A Bayesian Neural Network based on Dropout Regulation
  Bayesian Neural Networks (BNN) have recently emerged in the Deep Learning
world for dealing with uncertainty estimation in classification tasks, and are
used in many application domains such as astrophysics, autonomous driving...BNN
assume a prior over the weights of a neural network instead of point estimates,
enabling in this way the estimation of both aleatoric and epistemic uncertainty
of the model prediction.Moreover, a particular type of BNN, namely MC Dropout,
assumes a Bernoulli distribution on the weights by using Dropout.Several
attempts to optimize the dropout rate exist, e.g. using a variational
approach.In this paper, we present a new method called "Dropout Regulation"
(DR), which consists of automatically adjusting the dropout rate during
training using a controller as used in automation.DR allows for a precise
estimation of the uncertainty which is comparable to the state-of-the-art while
remaining simple to implement.
