Prior choice affects ability of Bayesian neural networks to identify
  unknowns
  Deep Bayesian neural networks (BNNs) are a powerful tool, though
computationally demanding, to perform parameter estimation while jointly
estimating uncertainty around predictions. BNNs are typically implemented using
arbitrary normal-distributed prior distributions on the model parameters. Here,
we explore the effects of different prior distributions on classification tasks
in BNNs and evaluate the evidence supporting the predictions based on posterior
probabilities approximated by Markov Chain Monte Carlo sampling and by
computing Bayes factors. We show that the choice of priors has a substantial
impact on the ability of the model to confidently assign data to the correct
class (true positive rates). Prior choice also affects significantly the
ability of a BNN to identify out-of-distribution instances as unknown (false
positive rates). When comparing our results against neural networks (NN) with
Monte Carlo dropout we found that BNNs generally outperform NNs. Finally, in
our tests we did not find a single best choice as prior distribution. Instead,
each dataset yielded the best results under a different prior, indicating that
testing alternative options can improve the performance of BNNs.
