Out-of-distribution Generalization in the Presence of Nuisance-Induced
  Spurious Correlations
  In many prediction problems, spurious correlations are induced by a changing
relationship between the label and a nuisance variable that is also correlated
with the covariates. For example, in classifying animals in natural images, the
background, which is a nuisance, can predict the type of animal. This
nuisance-label relationship does not always hold, and the performance of a
model trained under one such relationship may be poor on data with a different
nuisance-label relationship. To build predictive models that perform well
regardless of the nuisance-label relationship, we develop Nuisance-Randomized
Distillation (NURD). We introduce the nuisance-randomized distribution, a
distribution where the nuisance and the label are independent. Under this
distribution, we define the set of representations such that conditioning on
any member, the nuisance and the label remain independent. We prove that the
representations in this set always perform better than chance, while
representations outside of this set may not. NURD finds a representation from
this set that is most informative of the label under the nuisance-randomized
distribution, and we prove that this representation achieves the highest
performance regardless of the nuisance-label relationship. We evaluate NURD on
several tasks including chest X-ray classification where, using non-lung
patches as the nuisance, NURD produces models that predict pneumonia under
strong spurious correlations.
