{'fraction_non_alphanumeric': 0.02081268582755203, 'fraction_numerical': 0.0, 'mean_word_length': 5.558441558441558, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Overview of Natural Language Processing Technologies and Rationales in Application

In the past decade, rapid advancement of new technologies including data technology, virtual reality (VR) and artificial intelligence (AI), which are all related to language disciplines, brings a new era of data-based language studies, relying on AI to enhance the language ability and VI to create fresh new experience. Practice of language processing in language disciplines by those technologies in turn promotes the emergence of some other revolutionary technologies, for example, the increasingly common data thinking and computational thinking in language research. In this context, it is of great significance to seize the opportunity of big data era, and make full use of AI and other new technologies to substantially promote language-related studies. Thus, an overview of several important language processing technologies and the corresponding rationales, as well as the latest progress is expounded in this paper.