Uncertainty Autoencoders: Learning Compressed Representations via
  Variational Information Maximization
  Compressed sensing techniques enable efficient acquisition and recovery of
sparse, high-dimensional data signals via low-dimensional projections. In this
work, we propose Uncertainty Autoencoders, a learning framework for
unsupervised representation learning inspired by compressed sensing. We treat
the low-dimensional projections as noisy latent representations of an
autoencoder and directly learn both the acquisition (i.e., encoding) and
amortized recovery (i.e., decoding) procedures. Our learning objective
optimizes for a tractable variational lower bound to the mutual information
between the datapoints and the latent representations. We show how our
framework provides a unified treatment to several lines of research in
dimensionality reduction, compressed sensing, and generative modeling.
Empirically, we demonstrate a 32% improvement on average over competing
approaches for the task of statistical compressed sensing of high-dimensional
datasets.
