Post-hoc Uncertainty Calibration for Domain Drift Scenarios
  We address the problem of uncertainty calibration. While standard deep neural
networks typically yield uncalibrated predictions, calibrated confidence scores
that are representative of the true likelihood of a prediction can be achieved
using post-hoc calibration methods. However, to date the focus of these
approaches has been on in-domain calibration. Our contribution is two-fold.
First, we show that existing post-hoc calibration methods yield highly
over-confident predictions under domain shift. Second, we introduce a simple
strategy where perturbations are applied to samples in the validation set
before performing the post-hoc calibration step. In extensive experiments, we
demonstrate that this perturbation step results in substantially better
calibration under domain shift on a wide range of architectures and modelling
tasks.
