Surrogate-assisted parallel tempering for Bayesian neural learning
  Due to the need for robust uncertainty quantification, Bayesian neural
learning has gained attention in the era of deep learning and big data. Markov
Chain Monte-Carlo (MCMC) methods typically implement Bayesian inference which
faces several challenges given a large number of parameters, complex and
multimodal posterior distributions, and computational complexity of large
neural network models. Parallel tempering MCMC addresses some of these
limitations given that they can sample multimodal posterior distributions and
utilize high-performance computing. However, certain challenges remain given
large neural network models and big data. Surrogate-assisted optimization
features the estimation of an objective function for models which are
computationally expensive. In this paper, we address the inefficiency of
parallel tempering MCMC for large-scale problems by combining parallel
computing features with surrogate assisted likelihood estimation that describes
the plausibility of a model parameter value, given specific observed data.
Hence, we present surrogate-assisted parallel tempering for Bayesian neural
learning for simple to computationally expensive models. Our results
demonstrate that the methodology significantly lowers the computational cost
while maintaining quality in decision making with Bayesian neural networks. The
method has applications for a Bayesian inversion and uncertainty quantification
for a broad range of numerical models.
