Active metric learning and classification using similarity queries
  Active learning is commonly used to train label-efficient models by
adaptively selecting the most informative queries. However, most active
learning strategies are designed to either learn a representation of the data
(e.g., embedding or metric learning) or perform well on a task (e.g.,
classification) on the data. However, many machine learning tasks involve a
combination of both representation learning and a task-specific goal. Motivated
by this, we propose a novel unified query framework that can be applied to any
problem in which a key component is learning a representation of the data that
reflects similarity. Our approach builds on similarity or nearest neighbor (NN)
queries which seek to select samples that result in improved embeddings. The
queries consist of a reference and a set of objects, with an oracle selecting
the object most similar (i.e., nearest) to the reference. In order to reduce
the number of solicited queries, they are chosen adaptively according to an
information theoretic criterion. We demonstrate the effectiveness of the
proposed strategy on two tasks -- active metric learning and active
classification -- using a variety of synthetic and real world datasets. In
particular, we demonstrate that actively selected NN queries outperform
recently developed active triplet selection methods in a deep metric learning
setting. Further, we show that in classification, actively selecting class
labels can be reformulated as a process of selecting the most informative NN
query, allowing direct application of our method.
