A Survey of Label-noise Representation Learning: Past, Present and
  Future
  Classical machine learning implicitly assumes that labels of the training
data are sampled from a clean distribution, which can be too restrictive for
real-world scenarios. However, statistical-learning-based methods may not train
deep learning models robustly with these noisy labels. Therefore, it is urgent
to design Label-Noise Representation Learning (LNRL) methods for robustly
training deep models with noisy labels. To fully understand LNRL, we conduct a
survey study. We first clarify a formal definition for LNRL from the
perspective of machine learning. Then, via the lens of learning theory and
empirical study, we figure out why noisy labels affect deep models'
performance. Based on the theoretical guidance, we categorize different LNRL
methods into three directions. Under this unified taxonomy, we provide a
thorough discussion of the pros and cons of different categories. More
importantly, we summarize the essential components of robust LNRL, which can
spark new directions. Lastly, we propose possible research directions within
LNRL, such as new datasets, instance-dependent LNRL, and adversarial LNRL. We
also envision potential directions beyond LNRL, such as learning with
feature-noise, preference-noise, domain-noise, similarity-noise, graph-noise
and demonstration-noise.
