{'fraction_non_alphanumeric': 0.01844532279314888, 'fraction_numerical': 0.0, 'mean_word_length': 4.801526717557252, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Model Checking a Rule-based Parser

In the field of natural language processing, one key difficulty for rule-based systems is the debugging and tuning of rules. In this paper we suggest a novel method that use model checking tools to theoretically verify the rule system of a feature-based parser. We discuss the modeling of the system: the establishment of Kripke structures, and most importantly, the method to compress the state space of the model. We use partial Kripke structures as a main tool for space compression. We show that the state space can be reduced considerably while the compressed model still keeps similar behavior as the non-compressed one. Initial examples of specifications are presented and certain restrictions on them are discussed.