Transfer Learning with Dynamic Distribution Adaptation
  Transfer learning aims to learn robust classifiers for the target domain by
leveraging knowledge from a source domain. Since the source and the target
domains are usually from different distributions, existing methods mainly focus
on adapting the cross-domain marginal or conditional distributions. However, in
real applications, the marginal and conditional distributions usually have
different contributions to the domain discrepancy. Existing methods fail to
quantitatively evaluate the different importance of these two distributions,
which will result in unsatisfactory transfer performance. In this paper, we
propose a novel concept called Dynamic Distribution Adaptation (DDA), which is
capable of quantitatively evaluating the relative importance of each
distribution. DDA can be easily incorporated into the framework of structural
risk minimization to solve transfer learning problems. On the basis of DDA, we
propose two novel learning algorithms: (1) Manifold Dynamic Distribution
Adaptation (MDDA) for traditional transfer learning, and (2) Dynamic
Distribution Adaptation Network (DDAN) for deep transfer learning. Extensive
experiments demonstrate that MDDA and DDAN significantly improve the transfer
learning performance and setup a strong baseline over the latest deep and
adversarial methods on digits recognition, sentiment analysis, and image
classification. More importantly, it is shown that marginal and conditional
distributions have different contributions to the domain divergence, and our
DDA is able to provide good quantitative evaluation of their relative
importance which leads to better performance. We believe this observation can
be helpful for future research in transfer learning.
