Approximate Sampling using an Accelerated Metropolis-Hastings based on
  Bayesian Optimization and Gaussian Processes
  Markov Chain Monte Carlo (MCMC) methods have a drawback when working with a
target distribution or likelihood function that is computationally expensive to
evaluate, specially when working with big data. This paper focuses on
Metropolis-Hastings (MH) algorithm for unimodal distributions. Here, an
enhanced MH algorithm is proposed that requires less number of expensive
function evaluations, has shorter burn-in period, and uses a better proposal
distribution. The main innovations include the use of Bayesian optimization to
reach the high probability region quickly, emulating the target distribution
using Gaussian processes (GP), and using Laplace approximation of the GP to
build a proposal distribution that captures the underlying correlation better.
The experiments show significant improvement over the regular MH. Statistical
comparison between the results from two algorithms is presented.
