Balancing Fairness and Robustness via Partial Invariance
  The Invariant Risk Minimization (IRM) framework aims to learn invariant
features from a set of environments for solving the out-of-distribution (OOD)
generalization problem. The underlying assumption is that the causal components
of the data generating distributions remain constant across the environments or
alternately, the data "overlaps" across environments to find meaningful
invariant features. Consequently, when the "overlap" assumption does not hold,
the set of truly invariant features may not be sufficient for optimal
prediction performance. Such cases arise naturally in networked settings and
hierarchical data-generating models, wherein the IRM performance becomes
suboptimal. To mitigate this failure case, we argue for a partial invariance
framework. The key idea is to introduce flexibility into the IRM framework by
partitioning the environments based on hierarchical differences, while
enforcing invariance locally within the partitions. We motivate this framework
in classification settings with causal distribution shifts across environments.
Our results show the capability of the partial invariant risk minimization to
alleviate the trade-off between fairness and risk in certain settings.
