Linear Regression Games: Convergence Guarantees to Approximate
  Out-of-Distribution Solutions
  Recently, invariant risk minimization (IRM) (Arjovsky et al.) was proposed as
a promising solution to address out-of-distribution (OOD) generalization. In
Ahuja et al., it was shown that solving for the Nash equilibria of a new class
of "ensemble-games" is equivalent to solving IRM. In this work, we extend the
framework in Ahuja et al. for linear regressions by projecting the
ensemble-game on an $\ell_{\infty}$ ball. We show that such projections help
achieve non-trivial OOD guarantees despite not achieving perfect invariance.
For linear models with confounders, we prove that Nash equilibria of these
games are closer to the ideal OOD solutions than the standard empirical risk
minimization (ERM) and we also provide learning algorithms that provably
converge to these Nash Equilibria. Empirical comparisons of the proposed
approach with the state-of-the-art show consistent gains in achieving OOD
solutions in several settings involving anti-causal variables and confounders.
