Merging Weak and Active Supervision for Semantic Parsing
  A semantic parser maps natural language commands (NLs) from the users to
executable meaning representations (MRs), which are later executed in certain
environment to obtain user-desired results. The fully-supervised training of
such parser requires NL/MR pairs, annotated by domain experts, which makes them
expensive to collect. However, weakly-supervised semantic parsers are learnt
only from pairs of NL and expected execution results, leaving the MRs latent.
While weak supervision is cheaper to acquire, learning from this input poses
difficulties. It demands that parsers search a large space with a very weak
learning signal and it is hard to avoid spurious MRs that achieve the correct
answer in the wrong way. These factors lead to a performance gap between
parsers trained in weakly- and fully-supervised setting. To bridge this gap, we
examine the intersection between weak supervision and active learning, which
allows the learner to actively select examples and query for manual annotations
as extra supervision to improve the model trained under weak supervision. We
study different active learning heuristics for selecting examples to query, and
various forms of extra supervision for such queries. We evaluate the
effectiveness of our method on two different datasets. Experiments on the
WikiSQL show that by annotating only 1.8% of examples, we improve over a
state-of-the-art weakly-supervised baseline by 6.4%, achieving an accuracy of
79.0%, which is only 1.3% away from the model trained with full supervision.
Experiments on WikiTableQuestions with human annotators show that our method
can improve the performance with only 100 active queries, especially for
weakly-supervised parsers learnt from a cold start.
