Hypothesis Testing for Class-Conditional Label Noise
  In this paper we provide machine learning practitioners with tools to answer
the question: is there class-conditional noise in my labels? In particular, we
present hypothesis tests to check whether a given dataset of instance-label
pairs has been corrupted with class-conditional label noise, as opposed to
uniform label noise, with the former biasing learning, while the latter --
under mild conditions -- does not. The outcome of these tests can then be used
in conjunction with other information to assess further steps. While previous
works explore the direct estimation of the noise rates, this is known to be
hard in practice and does not offer a real understanding of how trustworthy the
estimates are. These methods typically require anchor points -- examples whose
true posterior is either 0 or 1. Differently, in this paper we assume we have
access to a set of anchor points whose true posterior is approximately 1/2. The
proposed hypothesis tests are built upon the asymptotic properties of Maximum
Likelihood Estimators for Logistic Regression models. We establish the main
properties of the tests, including a theoretical and empirical analysis of the
dependence of the power on the test on the training sample size, the number of
anchor points, the difference of the noise rates and the use of relaxed
anchors.
