Meta-Learning Transferable Active Learning Policies by Deep
  Reinforcement Learning
  Active learning (AL) aims to enable training high performance classifiers
with low annotation cost by predicting which subset of unlabelled instances
would be most beneficial to label. The importance of AL has motivated extensive
research, proposing a wide variety of manually designed AL algorithms with
diverse theoretical and intuitive motivations. In contrast to this body of
research, we propose to treat active learning algorithm design as a
meta-learning problem and learn the best criterion from data. We model an
active learning algorithm as a deep neural network that inputs the base learner
state and the unlabelled point set and predicts the best point to annotate
next. Training this active query policy network with reinforcement learning,
produces the best non-myopic policy for a given dataset. The key challenge in
achieving a general solution to AL then becomes that of learner generalisation,
particularly across heterogeneous datasets. We propose a multi-task
dataset-embedding approach that allows dataset-agnostic active learners to be
trained. Our evaluation shows that AL algorithms trained in this way can
directly generalise across diverse problems.
