Combating Label Noise in Deep Learning Using Abstention
  We introduce a novel method to combat label noise when training deep neural
networks for classification. We propose a loss function that permits abstention
during training thereby allowing the DNN to abstain on confusing samples while
continuing to learn and improve classification performance on the non-abstained
samples. We show how such a deep abstaining classifier (DAC) can be used for
robust learning in the presence of different types of label noise. In the case
of structured or systematic label noise -- where noisy training labels or
confusing examples are correlated with underlying features of the data--
training with abstention enables representation learning for features that are
associated with unreliable labels. In the case of unstructured (arbitrary)
label noise, abstention during training enables the DAC to be used as an
effective data cleaner by identifying samples that are likely to have label
noise. We provide analytical results on the loss function behavior that enable
dynamic adaption of abstention rates based on learning progress during
training. We demonstrate the utility of the deep abstaining classifier for
various image classification tasks under different types of label noise; in the
case of arbitrary label noise, we show significant improvements over previously
published results on multiple image benchmarks. Source code is available at
https://github.com/thulas/dac-label-noise
