Low-confidence Samples Matter for Domain Adaptation
  Domain adaptation (DA) aims to transfer knowledge from a label-rich source
domain to a related but label-scarce target domain. The conventional DA
strategy is to align the feature distributions of the two domains. Recently,
increasing researches have focused on self-training or other semi-supervised
algorithms to explore the data structure of the target domain. However, the
bulk of them depend largely on confident samples in order to build reliable
pseudo labels, prototypes or cluster centers. Representing the target data
structure in such a way would overlook the huge low-confidence samples,
resulting in sub-optimal transferability that is biased towards the samples
similar to the source domain. To overcome this issue, we propose a novel
contrastive learning method by processing low-confidence samples, which
encourages the model to make use of the target data structure through the
instance discrimination process. To be specific, we create positive and
negative pairs only using low-confidence samples, and then re-represent the
original features with the classifier weights rather than directly utilizing
them, which can better encode the task-specific semantic information.
Furthermore, we combine cross-domain mixup to augment the proposed contrastive
loss. Consequently, the domain gap can be well bridged through contrastive
learning of intermediate representations across domains. We evaluate the
proposed method in both unsupervised and semi-supervised DA settings, and
extensive experimental results on benchmarks reveal that our method is
effective and achieves state-of-the-art performance. The code can be found in
https://github.com/zhyx12/MixLRCo.
