Learning by Self-Explanation, with Application to Neural Architecture
  Search
  Learning by self-explanation is an effective learning technique in human
learning, where students explain a learned topic to themselves for deepening
their understanding of this topic. It is interesting to investigate whether
this explanation-driven learning methodology broadly used by humans is helpful
for improving machine learning as well. Based on this inspiration, we propose a
novel machine learning method called learning by self-explanation (LeaSE). In
our approach, an explainer model improves its learning ability by trying to
clearly explain to an audience model regarding how a prediction outcome is
made. LeaSE is formulated as a four-level optimization problem involving a
sequence of four learning stages which are conducted end-to-end in a unified
framework: 1) explainer learns; 2) explainer explains; 3) audience learns; 4)
explainer re-learns based on the performance of the audience. We develop an
efficient algorithm to solve the LeaSE problem. We apply LeaSE for neural
architecture search on CIFAR-100, CIFAR-10, and ImageNet. Experimental results
strongly demonstrate the effectiveness of our method.
