Learning from Failure: Training Debiased Classifier from Biased
  Classifier
  Neural networks often learn to make predictions that overly rely on spurious
correlation existing in the dataset, which causes the model to be biased. While
previous work tackles this issue by using explicit labeling on the spuriously
correlated attributes or presuming a particular bias type, we instead utilize a
cheaper, yet generic form of human knowledge, which can be widely applicable to
various types of bias. We first observe that neural networks learn to rely on
the spurious correlation only when it is "easier" to learn than the desired
knowledge, and such reliance is most prominent during the early phase of
training. Based on the observations, we propose a failure-based debiasing
scheme by training a pair of neural networks simultaneously. Our main idea is
twofold; (a) we intentionally train the first network to be biased by
repeatedly amplifying its "prejudice", and (b) we debias the training of the
second network by focusing on samples that go against the prejudice of the
biased network in (a). Extensive experiments demonstrate that our method
significantly improves the training of the network against various types of
biases in both synthetic and real-world datasets. Surprisingly, our framework
even occasionally outperforms the debiasing methods requiring explicit
supervision of the spuriously correlated attributes.
