Towards a Robust Differentiable Architecture Search under Label Noise
  Neural Architecture Search (NAS) is the game changer in designing robust
neural architectures. Architectures designed by NAS outperform or compete with
the best manual network designs in terms of accuracy, size, memory footprint
and FLOPs. That said, previous studies focus on developing NAS algorithms for
clean high quality data, a restrictive and somewhat unrealistic assumption. In
this paper, focusing on the differentiable NAS algorithms, we show that vanilla
NAS algorithms suffer from a performance loss if class labels are noisy. To
combat this issue, we make use of the principle of information bottleneck as a
regularizer. This leads us to develop a noise injecting operation that is
included during the learning process, preventing the network from learning from
noisy samples. Our empirical evaluations show that the noise injecting
operation does not degrade the performance of the NAS algorithm if the data is
indeed clean. In contrast, if the data is noisy, the architecture learned by
our algorithm comfortably outperforms algorithms specifically equipped with
sophisticated mechanisms to learn in the presence of label noise. In contrast
to many algorithms designed to work in the presence of noisy labels, prior
knowledge about the properties of the noise and its characteristics are not
required for our algorithm.
