Learning by Examples Based on Multi-level Optimization
  Learning by examples, which learns to solve a new problem by looking into how
similar problems are solved, is an effective learning method in human learning.
When a student learns a new topic, he/she finds out exemplar topics that are
similar to this new topic and studies the exemplar topics to deepen the
understanding of the new topic. We aim to investigate whether this powerful
learning skill can be borrowed from humans to improve machine learning as well.
In this work, we propose a novel learning approach called Learning By Examples
(LBE). Our approach automatically retrieves a set of training examples that are
similar to query examples and predicts labels for query examples by using class
labels of the retrieved examples. We propose a three-level optimization
framework to formulate LBE which involves three stages of learning: learning a
Siamese network to retrieve similar examples; learning a matching network to
make predictions on query examples by leveraging class labels of retrieved
similar examples; learning the ``ground-truth'' similarities between training
examples by minimizing the validation loss. We develop an efficient algorithm
to solve the LBE problem and conduct extensive experiments on various
benchmarks where the results demonstrate the effectiveness of our method on
both supervised and few-shot learning.
