Learning from Mistakes -- A Framework for Neural Architecture Search
  Learning from one's mistakes is an effective human learning technique where
the learners focus more on the topics where mistakes were made, so as to deepen
their understanding. In this paper, we investigate if this human learning
strategy can be applied in machine learning. We propose a novel machine
learning method called Learning From Mistakes (LFM), wherein the learner
improves its ability to learn by focusing more on the mistakes during revision.
We formulate LFM as a three-stage optimization problem: 1) learner learns; 2)
learner re-learns focusing on the mistakes, and; 3) learner validates its
learning. We develop an efficient algorithm to solve the LFM problem. We apply
the LFM framework to neural architecture search on CIFAR-10, CIFAR-100, and
Imagenet. Experimental results strongly demonstrate the effectiveness of our
model.
