Robust contrastive learning and nonlinear ICA in the presence of
  outliers
  Nonlinear independent component analysis (ICA) is a general framework for
unsupervised representation learning, and aimed at recovering the latent
variables in data. Recent practical methods perform nonlinear ICA by solving a
series of classification problems based on logistic regression. However, it is
well-known that logistic regression is vulnerable to outliers, and thus the
performance can be strongly weakened by outliers. In this paper, we first
theoretically analyze nonlinear ICA models in the presence of outliers. Our
analysis implies that estimation in nonlinear ICA can be seriously hampered
when outliers exist on the tails of the (noncontaminated) target density, which
happens in a typical case of contamination by outliers. We develop two robust
nonlinear ICA methods based on the {\gamma}-divergence, which is a robust
alternative to the KL-divergence in logistic regression. The proposed methods
are shown to have desired robustness properties in the context of nonlinear
ICA. We also experimentally demonstrate that the proposed methods are very
robust and outperform existing methods in the presence of outliers. Finally,
the proposed method is applied to ICA-based causal discovery and shown to find
a plausible causal relationship on fMRI data.
