Generalizing Variational Autoencoders with Hierarchical Empirical Bayes
  Variational Autoencoders (VAEs) have experienced recent success as
data-generating models by using simple architectures that do not require
significant fine-tuning of hyperparameters. However, VAEs are known to suffer
from over-regularization which can lead to failure to escape local maxima. This
phenomenon, known as posterior collapse, prevents learning a meaningful latent
encoding of the data. Recent methods have mitigated this issue by
deterministically moment-matching an aggregated posterior distribution to an
aggregate prior. However, abandoning a probabilistic framework (and thus
relying on point estimates) can both lead to a discontinuous latent space and
generate unrealistic samples. Here we present Hierarchical Empirical Bayes
Autoencoder (HEBAE), a computationally stable framework for probabilistic
generative models. Our key contributions are two-fold. First, we make gains by
placing a hierarchical prior over the encoding distribution, enabling us to
adaptively balance the trade-off between minimizing the reconstruction loss
function and avoiding over-regularization. Second, we show that assuming a
general dependency structure between variables in the latent space produces
better convergence onto the mean-field assumption for improved posterior
inference. Overall, HEBAE is more robust to a wide-range of hyperparameter
initializations than an analogous VAE. Using data from MNIST and CelebA, we
illustrate the ability of HEBAE to generate higher quality samples based on FID
score than existing autoencoder-based approaches.
