{'fraction_non_alphanumeric': 0.0176678445229682, 'fraction_numerical': 0.0, 'mean_word_length': 4.785714285714286, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Comparing test-suite based evaluation and corpus-based evaluation of a wide-coverage grammar for English

In this paper we present our experiences in the evaluation of a wide-coverage grammar of English: the XTAG English grammar. We give a brief history of previous evaluations done using the XTAG grammar and then describe a pair of new evaluations done on a corpus of weather reports and the CSLI LKB test suite. Based on these experiments, we discuss the differing merits of naturally occurring corpora and test suites in the evaluation of wide-coverage grammars.