Increasing Robustness to Spurious Correlations using Forgettable
  Examples
  Neural NLP models tend to rely on spurious correlations between labels and
input features to perform their tasks. Minority examples, i.e., examples that
contradict the spurious correlations present in the majority of data points,
have been shown to increase the out-of-distribution generalization of
pre-trained language models. In this paper, we first propose using example
forgetting to find minority examples without prior knowledge of the spurious
correlations present in the dataset. Forgettable examples are instances either
learned and then forgotten during training or never learned. We empirically
show how these examples are related to minorities in our training sets. Then,
we introduce a new approach to robustify models by fine-tuning our models
twice, first on the full training data and second on the minorities only. We
obtain substantial improvements in out-of-distribution generalization when
applying our approach to the MNLI, QQP, and FEVER datasets.
