Generation and Analysis of Feature-Dependent Pseudo Noise for Training
  Deep Neural Networks
  Training Deep neural networks (DNNs) on noisy labeled datasets is a
challenging problem, because learning on mislabeled examples deteriorates the
performance of the network. As the ground truth availability is limited with
real-world noisy datasets, previous papers created synthetic noisy datasets by
randomly modifying the labels of training examples of clean datasets. However,
no final conclusions can be derived by just using this random noise, since it
excludes feature-dependent noise. Thus, it is imperative to generate
feature-dependent noisy datasets that additionally provide ground truth.
Therefore, we propose an intuitive approach to creating feature-dependent noisy
datasets by utilizing the training predictions of DNNs on clean datasets that
also retain true label information. We refer to these datasets as "Pseudo Noisy
datasets". We conduct several experiments to establish that Pseudo noisy
datasets resemble feature-dependent noisy datasets across different conditions.
We further randomly generate synthetic noisy datasets with the same noise
distribution as that of Pseudo noise (referred as "Randomized Noise") to
empirically show that i) learning is easier with feature-dependent label noise
compared to random noise, ii) irrespective of noise distribution, Pseudo noisy
datasets mimic feature-dependent label noise and iii) current training methods
are not generalizable to feature-dependent label noise. Therefore, we believe
that Pseudo noisy datasets will be quite helpful to study and develop robust
training methods.
