Reliable Uncertainties for Bayesian Neural Networks using
  Alpha-divergences
  Bayesian Neural Networks (BNNs) often result uncalibrated after training,
usually tending towards overconfidence. Devising effective calibration methods
with low impact in terms of computational complexity is thus of central
interest. In this paper we present calibration methods for BNNs based on the
alpha divergences from Information Geometry. We compare the use of alpha
divergence in training and in calibration, and we show how the use in
calibration provides better calibrated uncertainty estimates for specific
choices of alpha and is more efficient especially for complex network
architectures. We empirically demonstrate the advantages of alpha calibration
in regression problems involving parameter estimation and inferred correlations
between output uncertainties.
