Derivative Manipulation for General Example Weighting
  Real-world large-scale datasets usually contain noisy labels and are
imbalanced. Therefore, we propose derivative manipulation (DM), a novel and
general example weighting approach for training robust deep models under these
adverse conditions.
  DM has two main merits. First, loss function and example weighting are common
techniques in the literature. DM reveals their connection (a loss function does
example weighting) and is a replacement of both. Second, despite that a loss
defines an example weighting scheme by its derivative, in the loss design, we
need to consider whether it is differentiable. Instead, DM is more flexible by
directly modifying the derivative so that a loss can be a non-elementary format
too. Technically, DM defines an emphasis density function by a derivative
magnitude function. DM is generic in that diverse weighting schemes can be
derived.
  Extensive experiments on both vision and language tasks prove DM's
effectiveness.
