{'fraction_non_alphanumeric': 0.024746906636670417, 'fraction_numerical': 0.0, 'mean_word_length': 5.223776223776224, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Linguistic and Behavioural Studies of Chinese Chat Language

Real-time communication platforms such as ICQ, MSN and online chat rooms are popular today over the Internet. The language used on these platforms differs significantly from standard natural language. Comparatively, this language, referred to as chat language, is informal, anomalous and dynamic. In this paper, the NIL corpus — a Chinese chat language text collection — is presented which is constructed to facilitate language study. Also, linguistic characteristics and ecological behaviour of the language are studied. We show that chat language is anomalous and dynamic in nature leading to the out-of-vocabulary, ambiguity and sparse data problems. To tackle these problems, a novel chat language model is proposed. The model is based on phonetic mapping between chat terms and their standard natural language counterparts.