How to Evaluate Uncertainty Estimates in Machine Learning for
  Regression?
  As neural networks become more popular, the need for accompanying uncertainty
estimates increases. The current testing methodology focusses on how good the
predictive uncertainty estimates explain the differences between predictions
and observations in a previously unseen test set. Intuitively this is a logical
approach. The current setup of benchmark data sets also allows easy comparison
between the different methods. We demonstrate, however, through both
theoretical arguments and simulations that this way of evaluating the quality
of uncertainty estimates has serious flaws. Firstly, it cannot disentangle the
aleatoric from the epistemic uncertainty. Secondly, the current methodology
considers the uncertainty averaged over all test samples, implicitly averaging
out overconfident and underconfident predictions. When checking if the correct
fraction of test points falls inside prediction intervals, a good score on
average gives no guarantee that the intervals are sensible for individual
points. We demonstrate through practical examples that these effects can result
in favoring a method, based on the predictive uncertainty, that has undesirable
behaviour of the confidence intervals. Finally, we propose a simulation-based
testing approach that addresses these problems while still allowing easy
comparison between different methods.
