Iterative Amortized Inference
  Inference models are a key component in scaling variational inference to deep
latent variable models, most notably as encoder networks in variational
auto-encoders (VAEs). By replacing conventional optimization-based inference
with a learned model, inference is amortized over data examples and therefore
more computationally efficient. However, standard inference models are
restricted to direct mappings from data to approximate posterior estimates. The
failure of these models to reach fully optimized approximate posterior
estimates results in an amortization gap. We aim toward closing this gap by
proposing iterative inference models, which learn to perform inference
optimization through repeatedly encoding gradients. Our approach generalizes
standard inference models in VAEs and provides insight into several empirical
findings, including top-down inference techniques. We demonstrate the inference
optimization capabilities of iterative inference models and show that they
outperform standard inference models on several benchmark data sets of images
and text.
