DINE: Domain Adaptation from Single and Multiple Black-box Predictors
  To ease the burden of labeling, unsupervised domain adaptation (UDA) aims to
transfer knowledge in previous and related labeled datasets (sources) to a new
unlabeled dataset (target). Despite impressive progress, prior methods always
need to access the raw source data and develop data-dependent alignment
approaches to recognize the target samples in a transductive learning manner,
which may raise privacy concerns from source individuals. Several recent
studies resort to an alternative solution by exploiting the well-trained
white-box model from the source domain, yet, it may still leak the raw data
through generative adversarial learning. This paper studies a practical and
interesting setting for UDA, where only black-box source models (i.e., only
network predictions are available) are provided during adaptation in the target
domain. To solve this problem, we propose a new two-step knowledge adaptation
framework called DIstill and fine-tuNE (DINE). Taking into consideration the
target data structure, DINE first distills the knowledge from the source
predictor to a customized target model, then fine-tunes the distilled model to
further fit the target domain. Besides, neural networks are not required to be
identical across domains in DINE, even allowing effective adaptation on a
low-resource device. Empirical results on three UDA scenarios (i.e.,
single-source, multi-source, and partial-set) confirm that DINE achieves highly
competitive performance compared to state-of-the-art data-dependent approaches.
Code is available at \url{https://github.com/tim-learn/DINE/}.
