Learning Invariant Representations and Risks for Semi-supervised Domain
  Adaptation
  The success of supervised learning hinges on the assumption that the training
and test data come from the same underlying distribution, which is often not
valid in practice due to potential distribution shift. In light of this, most
existing methods for unsupervised domain adaptation focus on achieving
domain-invariant representations and small source domain error. However, recent
works have shown that this is not sufficient to guarantee good generalization
on the target domain, and in fact, is provably detrimental under label
distribution shift. Furthermore, in many real-world applications it is often
feasible to obtain a small amount of labeled data from the target domain and
use them to facilitate model training with source data. Inspired by the above
observations, in this paper we propose the first method that aims to
simultaneously learn invariant representations and risks under the setting of
semi-supervised domain adaptation (Semi-DA). First, we provide a finite sample
bound for both classification and regression problems under Semi-DA. The bound
suggests a principled way to obtain target generalization, i.e. by aligning
both the marginal and conditional distributions across domains in feature
space. Motivated by this, we then introduce the LIRR algorithm for jointly
\textbf{L}earning \textbf{I}nvariant \textbf{R}epresentations and
\textbf{R}isks. Finally, extensive experiments are conducted on both
classification and regression tasks, which demonstrates LIRR consistently
achieves state-of-the-art performance and significant improvements compared
with the methods that only learn invariant representations or invariant risks.
