BALanCe: Deep Bayesian Active Learning via Equivalence Class Annealing
  Active learning has demonstrated data efficiency in many fields. Existing
active learning algorithms, especially in the context of deep Bayesian active
models, rely heavily on the quality of uncertainty estimations of the model.
However, such uncertainty estimates could be heavily biased, especially with
limited and imbalanced training data. In this paper, we propose BALanCe, a
Bayesian deep active learning framework that mitigates the effect of such
biases. Concretely, BALanCe employs a novel acquisition function which
leverages the structure captured by equivalence hypothesis classes and
facilitates differentiation among different equivalence classes. Intuitively,
each equivalence class consists of instantiations of deep models with similar
predictions, and BALanCe adaptively adjusts the size of the equivalence classes
as learning progresses. Besides the fully sequential setting, we further
propose Batch-BALanCe -- a generalization of the sequential algorithm to the
batched setting -- to efficiently select batches of training examples that are
jointly effective for model improvement. We show that Batch-BALanCe achieves
state-of-the-art performance on several benchmark datasets for active learning,
and that both algorithms can effectively handle realistic challenges that often
involve multi-class and imbalanced data.
