Sequential Domain Adaptation by Synthesizing Distributionally Robust
  Experts
  Least squares estimators, when trained on a few target domain samples, may
predict poorly. Supervised domain adaptation aims to improve the predictive
accuracy by exploiting additional labeled training samples from a source
distribution that is close to the target distribution. Given available data, we
investigate novel strategies to synthesize a family of least squares estimator
experts that are robust with regard to moment conditions. When these moment
conditions are specified using Kullback-Leibler or Wasserstein-type
divergences, we can find the robust estimators efficiently using convex
optimization. We use the Bernstein online aggregation algorithm on the proposed
family of robust experts to generate predictions for the sequential stream of
target test samples. Numerical experiments on real data show that the robust
strategies may outperform non-robust interpolations of the empirical least
squares estimators.
