Surrogate Losses in Passive and Active Learning
  Active learning is a type of sequential design for supervised machine
learning, in which the learning algorithm sequentially requests the labels of
selected instances from a large pool of unlabeled data points. The objective is
to produce a classifier of relatively low risk, as measured under the 0-1 loss,
ideally using fewer label requests than the number of random labeled data
points sufficient to achieve the same. This work investigates the potential
uses of surrogate loss functions in the context of active learning.
Specifically, it presents an active learning algorithm based on an arbitrary
classification-calibrated surrogate loss function, along with an analysis of
the number of label requests sufficient for the classifier returned by the
algorithm to achieve a given risk under the 0-1 loss. Interestingly, these
results cannot be obtained by simply optimizing the surrogate risk via active
learning to an extent sufficient to provide a guarantee on the 0-1 loss, as is
common practice in the analysis of surrogate losses for passive learning. Some
of the results have additional implications for the use of surrogate losses in
passive learning.
