Principled Hybrids of Generative and Discriminative Domain Adaptation
  We propose a probabilistic framework for domain adaptation that blends both
generative and discriminative modeling in a principled way. Under this
framework, generative and discriminative models correspond to specific choices
of the prior over parameters. This provides us a very general way to
interpolate between generative and discriminative extremes through different
choices of priors. By maximizing both the marginal and the conditional
log-likelihoods, models derived from this framework can use both labeled
instances from the source domain as well as unlabeled instances from both
source and target domains. Under this framework, we show that the popular
reconstruction loss of autoencoder corresponds to an upper bound of the
negative marginal log-likelihoods of unlabeled instances, where marginal
distributions are given by proper kernel density estimations. This provides a
way to interpret the empirical success of autoencoders in domain adaptation and
semi-supervised learning. We instantiate our framework using neural networks,
and build a concrete model, DAuto. Empirically, we demonstrate the
effectiveness of DAuto on text, image and speech datasets, showing that it
outperforms related competitors when domain adaptation is possible.
