Bayesian Neural Network Ensembles
  Ensembles of neural networks (NNs) have long been used to estimate predictive
uncertainty; a small number of NNs are trained from different initialisations
and sometimes on differing versions of the dataset. The variance of the
ensemble's predictions is interpreted as its epistemic uncertainty. The appeal
of ensembling stems from being a collection of regular NNs - this makes them
both scalable and easily implementable. They have achieved strong empirical
results in recent years, often presented as a practical alternative to more
costly Bayesian NNs (BNNs). The departure from Bayesian methodology is of
concern since the Bayesian framework provides a principled, widely-accepted
approach to handling uncertainty. In this extended abstract we derive and
implement a modified NN ensembling scheme, which provides a consistent
estimator of the Bayesian posterior in wide NNs - regularising parameters about
values drawn from a prior distribution.
