Bayesian Inference with Anchored Ensembles of Neural Networks, and
  Application to Exploration in Reinforcement Learning
  The use of ensembles of neural networks (NNs) for the quantification of
predictive uncertainty is widespread. However, the current justification is
intuitive rather than analytical. This work proposes one minor modification to
the normal ensembling methodology, which we prove allows the ensemble to
perform Bayesian inference, hence converging to the corresponding Gaussian
Process as both the total number of NNs, and the size of each, tend to
infinity. This working paper provides early-stage results in a reinforcement
learning setting, analysing the practicality of the technique for an ensemble
of small, finite number. Using the uncertainty estimates produced by anchored
ensembles to govern the exploration-exploitation process results in steadier,
more stable learning.
