Generalized Data Weighting via Class-level Gradient Manipulation
  Label noise and class imbalance are two major issues coexisting in real-world
datasets. To alleviate the two issues, state-of-the-art methods reweight each
instance by leveraging a small amount of clean and unbiased data. Yet, these
methods overlook class-level information within each instance, which can be
further utilized to improve performance. To this end, in this paper, we propose
Generalized Data Weighting (GDW) to simultaneously mitigate label noise and
class imbalance by manipulating gradients at the class level. To be specific,
GDW unrolls the loss gradient to class-level gradients by the chain rule and
reweights the flow of each gradient separately. In this way, GDW achieves
remarkable performance improvement on both issues. Aside from the performance
gain, GDW efficiently obtains class-level weights without introducing any extra
computational cost compared with instance weighting methods. Specifically, GDW
performs a gradient descent step on class-level weights, which only relies on
intermediate gradients. Extensive experiments in various settings verify the
effectiveness of GDW. For example, GDW outperforms state-of-the-art methods by
$2.56\%$ under the $60\%$ uniform noise setting in CIFAR10. Our code is
available at https://github.com/GGchen1997/GDW-NIPS2021.
