An Architecture for Deep, Hierarchical Generative Models
  We present an architecture which lets us train deep, directed generative
models with many layers of latent variables. We include deterministic paths
between all latent variables and the generated output, and provide a richer set
of connections between computations for inference and generation, which enables
more effective communication of information throughout the model during
training. To improve performance on natural images, we incorporate a
lightweight autoregressive model in the reconstruction distribution. These
techniques permit end-to-end training of models with 10+ layers of latent
variables. Experiments show that our approach achieves state-of-the-art
performance on standard image modelling benchmarks, can expose latent class
structure in the absence of label information, and can provide convincing
imputations of occluded regions in natural images.
