Meta-Learned Invariant Risk Minimization
  Empirical Risk Minimization (ERM) based machine learning algorithms have
suffered from weak generalization performance on data obtained from
out-of-distribution (OOD). To address this problem, Invariant Risk Minimization
(IRM) objective was suggested to find invariant optimal predictor which is less
affected by the changes in data distribution. However, even with such progress,
IRMv1, the practical formulation of IRM, still shows performance degradation
when there are not enough training data, and even fails to generalize to OOD,
if the number of spurious correlations is larger than the number of
environments. In this paper, to address such problems, we propose a novel
meta-learning based approach for IRM. In this method, we do not assume the
linearity of classifier for the ease of optimization, and solve ideal bi-level
IRM objective with Model-Agnostic Meta-Learning (MAML) framework. Our method is
more robust to the data with spurious correlations and can provide an invariant
optimal classifier even when data from each distribution are scarce. In
experiments, we demonstrate that our algorithm not only has better OOD
generalization performance than IRMv1 and all IRM variants, but also addresses
the weakness of IRMv1 with improved stability.
