Learning to Split for Automatic Bias Detection
  Classifiers are biased when trained on biased datasets. As a remedy, we
propose Learning to Split (ls), an algorithm for automatic bias detection.
Given a dataset with input-label pairs, ls learns to split this dataset so that
predictors trained on the training split generalize poorly to the testing
split. This performance gap provides a proxy for measuring the degree of bias
in the learned features and can therefore be used to reduce biases. Identifying
non-generalizable splits is challenging as we don't have any explicit
annotations about how to split. In this work, we show that the prediction
correctness of the testing example can be used as a source of weak supervision:
generalization performance will drop if we move examples that are predicted
correctly away from the testing split, leaving only those that are
mispredicted. We evaluate our approach on Beer Review, Waterbirds, CelebA and
MNLI. Empirical results show that ls is able to generate astonishingly
challenging splits that correlate with human-identified biases. Moreover, we
demonstrate that combining robust learning algorithms (such as group DRO) with
splits identified by ls enables automatic de-biasing. Compared with previous
state-of-the-arts, we substantially improves the worst-group performance (23.4%
on average) when the source of biases is unknown during training and
validation.
