ALLWAS: Active Learning on Language models in WASserstein space
  Active learning has emerged as a standard paradigm in areas with scarcity of
labeled training data, such as in the medical domain. Language models have
emerged as the prevalent choice of several natural language tasks due to the
performance boost offered by these models. However, in several domains, such as
medicine, the scarcity of labeled training data is a common issue. Also, these
models may not work well in cases where class imbalance is prevalent. Active
learning may prove helpful in these cases to boost the performance with a
limited label budget. To this end, we propose a novel method using sampling
techniques based on submodular optimization and optimal transport for active
learning in language models, dubbed ALLWAS. We construct a sampling strategy
based on submodular optimization of the designed objective in the gradient
domain. Furthermore, to enable learning from few samples, we propose a novel
strategy for sampling from the Wasserstein barycenters. Our empirical
evaluations on standard benchmark datasets for text classification show that
our methods perform significantly better (>20% relative increase in some cases)
than existing approaches for active learning on language models.
