Adaptive Self-training for Few-shot Neural Sequence Labeling
  Sequence labeling is an important technique employed for many Natural
Language Processing (NLP) tasks, such as Named Entity Recognition (NER), slot
tagging for dialog systems and semantic parsing. Large-scale pre-trained
language models obtain very good performance on these tasks when fine-tuned on
large amounts of task-specific labeled data. However, such large-scale labeled
datasets are difficult to obtain for several tasks and domains due to the high
cost of human annotation as well as privacy and data access constraints for
sensitive user applications. This is exacerbated for sequence labeling tasks
requiring such annotations at token-level. In this work, we develop techniques
to address the label scarcity challenge for neural sequence labeling models.
Specifically, we develop self-training and meta-learning techniques for
training neural sequence taggers with few labels. While self-training serves as
an effective mechanism to learn from large amounts of unlabeled data --
meta-learning helps in adaptive sample re-weighting to mitigate error
propagation from noisy pseudo-labels. Extensive experiments on six benchmark
datasets including two for massive multilingual NER and four slot tagging
datasets for task-oriented dialog systems demonstrate the effectiveness of our
method. With only 10 labeled examples for each class for each task, our method
obtains 10% improvement over state-of-the-art systems demonstrating its
effectiveness for the low-resource setting.
