Dropout Strikes Back: Improved Uncertainty Estimation via Diversity
  Sampling
  Uncertainty estimation for machine learning models is of high importance in
many scenarios such as constructing the confidence intervals for model
predictions and detection of out-of-distribution or adversarially generated
points. In this work, we show that modifying the sampling distributions for
dropout layers in neural networks improves the quality of uncertainty
estimation. Our main idea consists of two main steps: computing data-driven
correlations between neurons and generating samples, which include maximally
diverse neurons. In a series of experiments on simulated and real-world data,
we demonstrate that the diversification via determinantal point processes-based
sampling achieves state-of-the-art results in uncertainty estimation for
regression and classification tasks. An important feature of our approach is
that it does not require any modification to the models or training procedures,
allowing straightforward application to any deep learning model with dropout
layers.
