Similarity Search for Efficient Active Learning and Search of Rare
  Concepts
  Many active learning and search approaches are intractable for large-scale
industrial settings with billions of unlabeled examples. Existing approaches
search globally for the optimal examples to label, scaling linearly or even
quadratically with the unlabeled data. In this paper, we improve the
computational efficiency of active learning and search methods by restricting
the candidate pool for labeling to the nearest neighbors of the currently
labeled set instead of scanning over all of the unlabeled data. We evaluate
several selection strategies in this setting on three large-scale computer
vision datasets: ImageNet, OpenImages, and a de-identified and aggregated
dataset of 10 billion images provided by a large internet company. Our approach
achieved similar mean average precision and recall as the traditional global
approach while reducing the computational cost of selection by up to three
orders of magnitude, thus enabling web-scale active learning.
