Cartography Active Learning
  We propose Cartography Active Learning (CAL), a novel Active Learning (AL)
algorithm that exploits the behavior of the model on individual instances
during training as a proxy to find the most informative instances for labeling.
CAL is inspired by data maps, which were recently proposed to derive insights
into dataset quality (Swayamdipta et al., 2020). We compare our method on
popular text classification tasks to commonly used AL strategies, which instead
rely on post-training behavior. We demonstrate that CAL is competitive to other
common AL methods, showing that training dynamics derived from small seed data
can be successfully used for AL. We provide insights into our new AL method by
analyzing batch-level statistics utilizing the data maps. Our results further
show that CAL results in a more data-efficient learning strategy, achieving
comparable or better results with considerably less training data.
