From Big to Small: Adaptive Learning to Partial-Set Domains
  Domain adaptation targets at knowledge acquisition and dissemination from a
labeled source domain to an unlabeled target domain under distribution shift.
Still, the common requirement of identical class space shared across domains
hinders applications of domain adaptation to partial-set domains. Recent
advances show that deep pre-trained models of large scale endow rich knowledge
to tackle diverse downstream tasks of small scale. Thus, there is a strong
incentive to adapt models from large-scale domains to small-scale domains. This
paper introduces Partial Domain Adaptation (PDA), a learning paradigm that
relaxes the identical class space assumption to that the source class space
subsumes the target class space. First, we present a theoretical analysis of
partial domain adaptation, which uncovers the importance of estimating the
transferable probability of each class and each instance across domains. Then,
we propose Selective Adversarial Network (SAN and SAN++) with a bi-level
selection strategy and an adversarial adaptation mechanism. The bi-level
selection strategy up-weighs each class and each instance simultaneously for
source supervised training, target self-training, and source-target adversarial
adaptation through the transferable probability estimated alternately by the
model. Experiments on standard partial-set datasets and more challenging tasks
with superclasses show that SAN++ outperforms several domain adaptation
methods.
