When is invariance useful in an Out-of-Distribution Generalization
  problem ?
  The goal of Out-of-Distribution (OOD) generalization problem is to train a
predictor that generalizes on all environments. Popular approaches in this
field use the hypothesis that such a predictor shall be an \textit{invariant
predictor} that captures the mechanism that remains constant across
environments. While these approaches have been experimentally successful in
various case studies, there is still much room for the theoretical validation
of this hypothesis. This paper presents a new set of theoretical conditions
necessary for an invariant predictor to achieve the OOD optimality. Our theory
not only applies to non-linear cases, but also generalizes the necessary
condition used in \citet{rojas2018invariant}. We also derive Inter Gradient
Alignment algorithm from our theory and demonstrate its competitiveness on
MNIST-derived benchmark datasets as well as on two of the three
\textit{Invariance Unit Tests} proposed by \citet{aubinlinear}.
