Surrogate Likelihoods for Variational Annealed Importance Sampling
  Variational inference is a powerful paradigm for approximate Bayesian
inference with a number of appealing properties, including support for model
learning and data subsampling. By contrast MCMC methods like Hamiltonian Monte
Carlo do not share these properties but remain attractive since, contrary to
parametric methods, MCMC is asymptotically unbiased. For these reasons
researchers have sought to combine the strengths of both classes of algorithms,
with recent approaches coming closer to realizing this vision in practice.
However, supporting data subsampling in these hybrid methods can be a
challenge, a shortcoming that we address by introducing a surrogate likelihood
that can be learned jointly with other variational parameters. We argue
theoretically that the resulting algorithm permits the user to make an
intuitive trade-off between inference fidelity and computational cost. In an
extensive empirical comparison we show that our method performs well in
practice and that it is well-suited for black-box inference in probabilistic
programming frameworks.
