Variational Bayes under Model Misspecification
  Variational Bayes (VB) is a scalable alternative to Markov chain Monte Carlo
(MCMC) for Bayesian posterior inference. Though popular, VB comes with few
theoretical guarantees, most of which focus on well-specified models. However,
models are rarely well-specified in practice. In this work, we study VB under
model misspecification. We prove the VB posterior is asymptotically normal and
centers at the value that minimizes the Kullback-Leibler (KL) divergence to the
true data-generating distribution. Moreover, the VB posterior mean centers at
the same value and is also asymptotically normal. These results generalize the
variational Bernstein--von Mises theorem [29] to misspecified models. As a
consequence of these results, we find that the model misspecification error
dominates the variational approximation error in VB posterior predictive
distributions. It explains the widely observed phenomenon that VB achieves
comparable predictive accuracy with MCMC even though VB uses an approximating
family. As illustrations, we study VB under three forms of model
misspecification, ranging from model over-/under-dispersion to latent
dimensionality misspecification. We conduct two simulation studies that
demonstrate the theoretical results.
