Identifiable Deep Generative Models via Sparse Decoding
  We develop the sparse VAE for unsupervised representation learning on
high-dimensional data. The sparse VAE learns a set of latent factors
(representations) which summarize the associations in the observed data
features. The underlying model is sparse in that each observed feature (i.e.
each dimension of the data) depends on a small subset of the latent factors. As
examples, in ratings data each movie is only described by a few genres; in text
data each word is only applicable to a few topics; in genomics, each gene is
active in only a few biological processes. We prove such sparse deep generative
models are identifiable: with infinite data, the true model parameters can be
learned. (In contrast, most deep generative models are not identifiable.) We
empirically study the sparse VAE with both simulated and real data. We find
that it recovers meaningful latent factors and has smaller heldout
reconstruction error than related methods.
