Learning to Reweight Examples for Robust Deep Learning
  Deep neural networks have been shown to be very powerful modeling tools for
many supervised learning tasks involving complex input patterns. However, they
can also easily overfit to training set biases and label noises. In addition to
various regularizers, example reweighting algorithms are popular solutions to
these problems, but they require careful tuning of additional hyperparameters,
such as example mining schedules and regularization hyperparameters. In
contrast to past reweighting methods, which typically consist of functions of
the cost value of each example, in this work we propose a novel meta-learning
algorithm that learns to assign weights to training examples based on their
gradient directions. To determine the example weights, our method performs a
meta gradient descent step on the current mini-batch example weights (which are
initialized from zero) to minimize the loss on a clean unbiased validation set.
Our proposed method can be easily implemented on any type of deep network, does
not require any additional hyperparameter tuning, and achieves impressive
performance on class imbalance and corrupted label problems where only a small
amount of clean validation data is available.
