Bridging the Theoretical Bound and Deep Algorithms for Open Set Domain
  Adaptation
  In the unsupervised open set domain adaptation (UOSDA), the target domain
contains unknown classes that are not observed in the source domain.
Researchers in this area aim to train a classifier to accurately: 1) recognize
unknown target data (data with unknown classes) and, 2) classify other target
data. To achieve this aim, a previous study has proven an upper bound of the
target-domain risk, and the open set difference, as an important term in the
upper bound, is used to measure the risk on unknown target data. By minimizing
the upper bound, a shallow classifier can be trained to achieve the aim.
However, if the classifier is very flexible (e.g., deep neural networks
(DNNs)), the open set difference will converge to a negative value when
minimizing the upper bound, which causes an issue where most target data are
recognized as unknown data. To address this issue, we propose a new upper bound
of target-domain risk for UOSDA, which includes four terms: source-domain risk,
$\epsilon$-open set difference ($\Delta_\epsilon$), a distributional
discrepancy between domains, and a constant. Compared to the open set
difference, $\Delta_\epsilon$ is more robust against the issue when it is being
minimized, and thus we are able to use very flexible classifiers (i.e., DNNs).
Then, we propose a new principle-guided deep UOSDA method that trains DNNs via
minimizing the new upper bound. Specifically, source-domain risk and
$\Delta_\epsilon$ are minimized by gradient descent, and the distributional
discrepancy is minimized via a novel open-set conditional adversarial training
strategy. Finally, compared to existing shallow and deep UOSDA methods, our
method shows the state-of-the-art performance on several benchmark datasets,
including digit recognition (MNIST, SVHN, USPS), object recognition (Office-31,
Office-Home), and face recognition (PIE).
