Active Learning by Acquiring Contrastive Examples
  Common acquisition functions for active learning use either uncertainty or
diversity sampling, aiming to select difficult and diverse data points from the
pool of unlabeled data, respectively. In this work, leveraging the best of both
worlds, we propose an acquisition function that opts for selecting
\textit{contrastive examples}, i.e. data points that are similar in the model
feature space and yet the model outputs maximally different predictive
likelihoods. We compare our approach, CAL (Contrastive Active Learning), with a
diverse set of acquisition functions in four natural language understanding
tasks and seven datasets. Our experiments show that CAL performs consistently
better or equal than the best performing baseline across all tasks, on both
in-domain and out-of-domain data. We also conduct an extensive ablation study
of our method and we further analyze all actively acquired datasets showing
that CAL achieves a better trade-off between uncertainty and diversity compared
to other strategies.
