Light-weight Calibrator: a Separable Component for Unsupervised Domain
  Adaptation
  Existing domain adaptation methods aim at learning features that can be
generalized among domains. These methods commonly require to update source
classifier to adapt to the target domain and do not properly handle the trade
off between the source domain and the target domain. In this work, instead of
training a classifier to adapt to the target domain, we use a separable
component called data calibrator to help the fixed source classifier recover
discrimination power in the target domain, while preserving the source domain's
performance. When the difference between two domains is small, the source
classifier's representation is sufficient to perform well in the target domain
and outperforms GAN-based methods in digits. Otherwise, the proposed method can
leverage synthetic images generated by GANs to boost performance and achieve
state-of-the-art performance in digits datasets and driving scene semantic
segmentation. Our method empirically reveals that certain intriguing hints,
which can be mitigated by adversarial attack to domain discriminators, are one
of the sources for performance degradation under the domain shift.
