Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive
  Learning
  Nonlinear ICA is a fundamental problem for unsupervised representation
learning, emphasizing the capacity to recover the underlying latent variables
generating the data (i.e., identifiability). Recently, the very first
identifiability proofs for nonlinear ICA have been proposed, leveraging the
temporal structure of the independent components. Here, we propose a general
framework for nonlinear ICA, which, as a special case, can make use of temporal
structure. It is based on augmenting the data by an auxiliary variable, such as
the time index, the history of the time series, or any other available
information. We propose to learn nonlinear ICA by discriminating between true
augmented data, or data in which the auxiliary variable has been randomized.
This enables the framework to be implemented algorithmically through logistic
regression, possibly in a neural network. We provide a comprehensive proof of
the identifiability of the model as well as the consistency of our estimation
method. The approach not only provides a general theoretical framework
combining and generalizing previously proposed nonlinear ICA models and
algorithms, but also brings practical advantages.
