Reducing the Long Tail Losses in Scientific Emulations with Active
  Learning
  Deep-learning-based models are increasingly used to emulate scientific
simulations to accelerate scientific research. However, accurate, supervised
deep learning models require huge amount of labelled data, and that often
becomes the bottleneck in employing neural networks. In this work, we leveraged
an active learning approach called core-set selection to actively select data,
per a pre-defined budget, to be labelled for training. To further improve the
model performance and reduce the training costs, we also warm started the
training using a shrink-and-perturb trick. We tested on two case studies in
different fields, namely galaxy halo occupation distribution modelling in
astrophysics and x-ray emission spectroscopy in plasma physics, and the results
are promising: we achieved competitive overall performance compared to using a
random sampling baseline, and more importantly, successfully reduced the larger
absolute losses, i.e. the long tail in the loss distribution, at virtually no
overhead costs.
