Improving Predictive Uncertainty Estimation using Dropout -- Hamiltonian
  Monte Carlo
  Estimating predictive uncertainty is crucial for many computer vision tasks,
from image classification to autonomous driving systems. Hamiltonian Monte
Carlo (HMC) is an sampling method for performing Bayesian inference. On the
other hand, Dropout regularization has been proposed as an approximate model
averaging technique that tends to improve generalization in large scale models
such as deep neural networks. Although, HMC provides convergence guarantees for
most standard Bayesian models, it does not handle discrete parameters arising
from Dropout regularization. In this paper, we present a robust methodology for
improving predictive uncertainty in classification problems, based on Dropout
and Hamiltonian Monte Carlo. Even though Dropout induces a non-smooth energy
function with no such convergence guarantees, the resulting discretization of
the Hamiltonian proves empirical success. The proposed method allows to
effectively estimate the predictive accuracy and to provide better
generalization for difficult test examples.
