On the Robustness of Monte Carlo Dropout Trained with Noisy Labels
  The memorization effect of deep learning hinders its performance to
effectively generalize on test set when learning with noisy labels. Prior study
has discovered that epistemic uncertainty techniques are robust when trained
with noisy labels compared with neural networks without uncertainty estimation.
They obtain prolonged memorization effect and better generalization performance
under the adversarial setting of noisy labels. Due to its superior performance
amongst other selected epistemic uncertainty methods under noisy labels, we
focus on Monte Carlo Dropout (MCDropout) and investigate why it is robust when
trained with noisy labels. Through empirical studies on datasets MNIST,
CIFAR-10, Animal-10n, we deep dive into three aspects of MCDropout under noisy
label setting: 1. efficacy: understanding the learning behavior and test
accuracy of MCDropout when training set contains artificially generated or
naturally embedded label noise; 2. representation volatility: studying the
responsiveness of neurons by examining the mean and standard deviation on each
neuron's activation; 3. network sparsity: investigating the network support of
MCDropout in comparison with deterministic neural networks. Our findings
suggest that MCDropout further sparsifies and regularizes the deterministic
neural networks and thus provides higher robustness against noisy labels.
