Adaptive Rule Discovery for Labeling Text Data
  Creating and collecting labeled data is one of the major bottlenecks in
machine learning pipelines and the emergence of automated feature generation
techniques such as deep learning, which typically requires a lot of training
data, has further exacerbated the problem. While weak-supervision techniques
have circumvented this bottleneck, existing frameworks either require users to
write a set of diverse, high-quality rules to label data (e.g., Snorkel), or
require a labeled subset of the data to automatically mine rules (e.g., Snuba).
The process of manually writing rules can be tedious and time consuming. At the
same time, creating a labeled subset of the data can be costly and even
infeasible in imbalanced settings. This is due to the fact that a random sample
in imbalanced settings often contains only a few positive instances.
  To address these shortcomings, we present Darwin, an interactive system
designed to alleviate the task of writing rules for labeling text data in
weakly-supervised settings. Given an initial labeling rule, Darwin
automatically generates a set of candidate rules for the labeling task at hand,
and utilizes the annotator's feedback to adapt the candidate rules. We describe
how Darwin is scalable and versatile. It can operate over large text corpora
(i.e., more than 1 million sentences) and supports a wide range of labeling
functions (i.e., any function that can be specified using a context free
grammar). Finally, we demonstrate with a suite of experiments over five
real-world datasets that Darwin enables annotators to generate
weakly-supervised labels efficiently and with a small cost. In fact, our
experiments show that rules discovered by Darwin on average identify 40% more
positive instances compared to Snuba even when it is provided with 1000 labeled
instances.
