On Deep Domain Adaptation: Some Theoretical Understandings
  Compared with shallow domain adaptation, recent progress in deep domain
adaptation has shown that it can achieve higher predictive performance and
stronger capacity to tackle structural data (e.g., image and sequential data).
The underlying idea of deep domain adaptation is to bridge the gap between
source and target domains in a joint space so that a supervised classifier
trained on labeled source data can be nicely transferred to the target domain.
This idea is certainly intuitive and powerful, however, limited theoretical
understandings have been developed to support its underpinning principle. In
this paper, we have provided a rigorous framework to explain why it is possible
to close the gap of the target and source domains in the joint space. More
specifically, we first study the loss incurred when performing transfer
learning from the source to the target domain. This provides a theory that
explains and generalizes existing work in deep domain adaptation which was
mainly empirical. This enables us to further explain why closing the gap in the
joint space can directly minimize the loss incurred for transfer learning
between the two domains. To our knowledge, this offers the first theoretical
result that characterizes a direct bound on the joint space and the gain of
transfer learning via deep domain adaptation
