Can Less be More? When Increasing-to-Balancing Label Noise Rates
  Considered Beneficial
  In this paper, we answer the question of when inserting label noise (less
informative labels) can instead return us more accurate and fair models. We are
primarily inspired by three observations: 1) In contrast to reducing label
noise rates, increasing the noise rates is easy to implement; 2) Increasing a
certain class of instances' label noise to balance the noise rates
(increasing-to-balancing) results in an easier learning problem; 3)
Increasing-to-balancing improves fairness guarantees against label bias. In
this paper, we first quantify the trade-offs introduced by increasing a certain
group of instances' label noise rate w.r.t. the loss of label informativeness
and the lowered learning difficulties. We analytically demonstrate when such an
increase is beneficial, in terms of either improved generalization power or the
fairness guarantees. Then we present a method to insert label noise properly
for the task of learning with noisy labels, either without or with a fairness
constraint. The primary technical challenge we face is due to the fact that we
would not know which data instances are suffering from higher noise, and we
would not have the ground truth labels to verify any possible hypothesis. We
propose a detection method that informs us which group of labels might suffer
from higher noise without using ground truth labels. We formally establish the
effectiveness of the proposed solution and demonstrate it with extensive
experiments.
