The Information Sieve
  We introduce a new framework for unsupervised learning of representations
based on a novel hierarchical decomposition of information. Intuitively, data
is passed through a series of progressively fine-grained sieves. Each layer of
the sieve recovers a single latent factor that is maximally informative about
multivariate dependence in the data. The data is transformed after each pass so
that the remaining unexplained information trickles down to the next layer.
Ultimately, we are left with a set of latent factors explaining all the
dependence in the original data and remainder information consisting of
independent noise. We present a practical implementation of this framework for
discrete variables and apply it to a variety of fundamental tasks in
unsupervised learning including independent component analysis, lossy and
lossless compression, and predicting missing values in data.
