Dropout can Simulate Exponential Number of Models for Sample Selection
  Techniques
  Following Coteaching, generally in the literature, two models are used in
sample selection based approaches for training with noisy labels. Meanwhile, it
is also well known that Dropout when present in a network trains an ensemble of
sub-networks. We show how to leverage this property of Dropout to train an
exponential number of shared models, by training a single model with Dropout.
We show how we can modify existing two model-based sample selection
methodologies to use an exponential number of shared models. Not only is it
more convenient to use a single model with Dropout, but this approach also
combines the natural benefits of Dropout with that of training an exponential
number of models, leading to improved results.
