Finding lost DG: Explaining domain generalization via model complexity
  The domain generalization (DG) problem setting challenges a model trained on
multiple known data distributions to generalise well on unseen data
distributions. Due to its practical importance, a large number of methods have
been proposed to address this challenge. However much of the work in general
purpose DG is heuristically motivated, as the DG problem is hard to model
formally; and recent evaluations have cast doubt on existing methods' practical
efficacy -- in particular compared to a well tuned empirical risk minimisation
baseline. We present a novel learning-theoretic generalisation bound for DG
that bounds unseen domain generalisation performance in terms of the model's
Rademacher complexity. Based on this, we conjecture that existing methods'
efficacy or lack thereof is largely determined by an empirical risk vs
predictor complexity trade-off, and demonstrate that their performance
variability can be explained in these terms. Algorithmically, this analysis
suggests that domain generalisation should be achieved by simply performing
regularised ERM with a leave-one-domain-out cross-validation objective.
Empirical results on the DomainBed benchmark corroborate this.
