Better than BERT but Worse than Baseline
  This paper compares BERT-SQuAD and Ab3P on the Abbreviation Definition
Identification (ADI) task. ADI inputs a text and outputs short forms
(abbreviations/acronyms) and long forms (expansions). BERT with reranking
improves over BERT without reranking but fails to reach the Ab3P rule-based
baseline. What is BERT missing? Reranking introduces two new features:
charmatch and freq. The first feature identifies opportunities to take
advantage of character constraints in acronyms and the second feature
identifies opportunities to take advantage of frequency constraints across
documents.
