Limited Gradient Descent: Learning With Noisy Labels
  Label noise may affect the generalization of classifiers, and the effective
learning of main patterns from samples with noisy labels is an important
challenge. Recent studies have shown that deep neural networks tend to
prioritize the learning of simple patterns over the memorization of noise
patterns. This suggests a possible method to search for the best generalization
that learns the main pattern until the noise begins to be memorized.
Traditional approaches often employ a clean validation set to find the best
stop timing of learning, i.e., early stopping. However, the generalization
performance of such methods relies on the quality of validation sets. Further,
in practice, a clean validation set is sometimes difficult to obtain. To solve
this problem, we propose a method that can estimate the optimal stopping timing
without a clean validation set, called limited gradient descent. We modified
the labels of a few samples in a noisy dataset to obtain false labels and to
create a reverse pattern. By monitoring the learning progress of the noisy and
reverse samples, we can determine the stop timing of learning. In this paper,
we also theoretically provide some necessary conditions on learning with noisy
labels. Experimental results on CIFAR-10 and CIFAR-100 datasets demonstrate
that our approach has a comparable generalization performance to methods
relying on a clean validation set. Thus, on the noisy Clothing-1M dataset, our
approach surpasses methods that rely on a clean validation set.
