Divide and Couple: Using Monte Carlo Variational Objectives for
  Posterior Approximation
  Recent work in variational inference (VI) uses ideas from Monte Carlo
estimation to tighten the lower bounds on the log-likelihood that are used as
objectives. However, there is no systematic understanding of how optimizing
different objectives relates to approximating the posterior distribution.
Developing such a connection is important if the ideas are to be applied to
inference-i.e., applications that require an approximate posterior and not just
an approximation of the log-likelihood. Given a VI objective defined by a Monte
Carlo estimator of the likelihood, we use a "divide and couple" procedure to
identify augmented proposal and target distributions. The divergence between
these is equal to the gap between the VI objective and the log-likelihood.
Thus, after maximizing the VI objective, the augmented variational distribution
may be used to approximate the posterior distribution.
