{'fraction_non_alphanumeric': 0.019328585961342827, 'fraction_numerical': 0.0, 'mean_word_length': 5.881118881118881, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Introduction to the Special Issue on Context in Natural Language Processing

Correct interpretation of natural language utterances and texts requires linguistic and nonlinguistic context. Text and dialogue processing are two application domains where the lack of good theories of context impedes significant progress in applying and developing new natural language processing (NLP) technologies. With today’s text processing technology, it is feasible to automatically create knowledge bases from fairly unconstrained texts such as newspaper archives. Ignoring context in such texts, however, results in knowledge bases that are not only very incomplete, but also dramatically different from knowledge bases created by humans, based on the same texts. Similarly, as speech technology matures, it becomes technically feasible to build dialogue systems. However, understanding dialogues, and especially multimodal dialogues, is not possible without some account of the role of context.