Learning by Passing Tests, with Application to Neural Architecture
  Search
  Learning through tests is a broadly used methodology in human learning and
shows great effectiveness in improving learning outcome: a sequence of tests
are made with increasing levels of difficulty; the learner takes these tests to
identify his/her weak points in learning and continuously addresses these weak
points to successfully pass these tests. We are interested in investigating
whether this powerful learning technique can be borrowed from humans to improve
the learning abilities of machines. We propose a novel learning approach called
learning by passing tests (LPT). In our approach, a tester model creates
increasingly more-difficult tests to evaluate a learner model. The learner
tries to continuously improve its learning ability so that it can successfully
pass however difficult tests created by the tester. We propose a multi-level
optimization framework to formulate LPT, where the tester learns to create
difficult and meaningful tests and the learner learns to pass these tests. We
develop an efficient algorithm to solve the LPT problem. Our method is applied
for neural architecture search and achieves significant improvement over
state-of-the-art baselines on CIFAR-100, CIFAR-10, and ImageNet.
