Robust Learning with the Hilbert-Schmidt Independence Criterion
  We investigate the use of a non-parametric independence measure, the
Hilbert-Schmidt Independence Criterion (HSIC), as a loss-function for learning
robust regression and classification models. This loss-function encourages
learning models where the distribution of the residuals between the label and
the model prediction is statistically independent of the distribution of the
instances themselves. This loss-function was first proposed by Mooij et al.
(2009) in the context of learning causal graphs. We adapt it to the task of
learning for unsupervised covariate shift: learning on a source domain without
access to any instances or labels from the unknown target domain, but with the
assumption that $p(y|x)$ (the conditional probability of labels given
instances) remains the same in the target domain. We show that the proposed
loss is expected to give rise to models that generalize well on a class of
target domains characterised by the complexity of their description within a
reproducing kernel Hilbert space. Experiments on unsupervised covariate shift
tasks demonstrate that models learned with the proposed loss-function
outperform models learned with standard loss functions, achieving
state-of-the-art results on a challenging cell-microscopy unsupervised
covariate shift task.
