Gradual Domain Adaptation in the Wild:When Intermediate Distributions
  are Absent
  We focus on the problem of domain adaptation when the goal is shifting the
model towards the target distribution, rather than learning domain invariant
representations. It has been shown that under the following two assumptions:
(a) access to samples from intermediate distributions, and (b) samples being
annotated with the amount of change from the source distribution, self-training
can be successfully applied on gradually shifted samples to adapt the model
toward the target distribution. We hypothesize having (a) is enough to enable
iterative self-training to slowly adapt the model to the target distribution,
by making use of an implicit curriculum. In the case where (a) does not hold,
we observe that iterative self-training falls short. We propose GIFT, a method
that creates virtual samples from intermediate distributions by interpolating
representations of examples from source and target domains. We evaluate an
iterative-self-training method on datasets with natural distribution shifts,
and show that when applied on top of other domain adaptation methods, it
improves the performance of the model on the target dataset. We run an analysis
on a synthetic dataset to show that in the presence of (a)
iterative-self-training naturally forms a curriculum of samples. Furthermore,
we show that when (a) does not hold, GIFT performs better than iterative
self-training.
