Amortized Population Gibbs Samplers with Neural Sufficient Statistics
  We develop amortized population Gibbs (APG) samplers, a class of scalable
methods that frames structured variational inference as adaptive importance
sampling. APG samplers construct high-dimensional proposals by iterating over
updates to lower-dimensional blocks of variables. We train each conditional
proposal by minimizing the inclusive KL divergence with respect to the
conditional posterior. To appropriately account for the size of the input data,
we develop a new parameterization in terms of neural sufficient statistics.
Experiments show that APG samplers can train highly structured deep generative
models in an unsupervised manner, and achieve substantial improvements in
inference accuracy relative to standard autoencoding variational methods.
