Decoupling "when to update" from "how to update"
  Deep learning requires data. A useful approach to obtain data is to be
creative and mine data from various sources, that were created for different
purposes. Unfortunately, this approach often leads to noisy labels. In this
paper, we propose a meta algorithm for tackling the noisy labels problem. The
key idea is to decouple "when to update" from "how to update". We demonstrate
the effectiveness of our algorithm by mining data for gender classification by
combining the Labeled Faces in the Wild (LFW) face recognition dataset with a
textual genderizing service, which leads to a noisy dataset. While our approach
is very simple to implement, it leads to state-of-the-art results. We analyze
some convergence properties of the proposed algorithm.
