Surprisingly Simple Semi-Supervised Domain Adaptation with Pretraining
  and Consistency
  Most modern unsupervised domain adaptation (UDA) approaches are rooted in
domain alignment, i.e., learning to align source and target features to learn a
target domain classifier using source labels. In semi-supervised domain
adaptation (SSDA), when the learner can access few target domain labels, prior
approaches have followed UDA theory to use domain alignment for learning. We
show that the case of SSDA is different and a good target classifier can be
learned without needing alignment. We use self-supervised pretraining (via
rotation prediction) and consistency regularization to achieve well separated
target clusters, aiding in learning a low error target classifier. With our
Pretraining and Consistency (PAC) approach, we achieve state of the art target
accuracy on this semi-supervised domain adaptation task, surpassing multiple
adversarial domain alignment methods, across multiple datasets. PAC, while
using simple techniques, performs remarkably well on large and challenging SSDA
benchmarks like DomainNet and Visda-17, often outperforming recent state of the
art by sizeable margins. Code for our experiments can be found at
https://github.com/venkatesh-saligrama/PAC
