Rethinking Distributional Matching Based Domain Adaptation
  Domain adaptation (DA) is a technique that transfers predictive models
trained on a labeled source domain to an unlabeled target domain, with the core
difficulty of resolving distributional shift between domains. Currently, most
popular DA algorithms are based on distributional matching (DM). However in
practice, realistic domain shifts (RDS) may violate their basic assumptions and
as a result these methods will fail. In this paper, in order to devise robust
DA algorithms, we first systematically analyze the limitations of DM based
methods, and then build new benchmarks with more realistic domain shifts to
evaluate the well-accepted DM methods. We further propose InstaPBM, a novel
Instance-based Predictive Behavior Matching method for robust DA. Extensive
experiments on both conventional and RDS benchmarks demonstrate both the
limitations of DM methods and the efficacy of InstaPBM: Compared with the best
baselines, InstaPBM improves the classification accuracy respectively by
$4.5\%$, $3.9\%$ on Digits5, VisDA2017, and $2.2\%$, $2.9\%$, $3.6\%$ on
DomainNet-LDS, DomainNet-ILDS, ID-TwO. We hope our intuitive yet effective
method will serve as a useful new direction and increase the robustness of DA
in real scenarios. Code will be available at anonymous link:
https://github.com/pikachusocute/InstaPBM-RobustDA.
