{'fraction_non_alphanumeric': 0.02865329512893983, 'fraction_numerical': 0.0, 'mean_word_length': 4.776859504132231, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Encoding identification of Internet packets

The rapid growth of the World Wide Web (WWW) has accelerated the push for using languages other than English. In particular, the East Asian languages, Chinese, Japanese, and Korean (CJK) are increasingly used over the Internet. In order to improve the quality of service, it becomes necessary to identify the language of the encoding types at a very low level. In this paper, a model based on a statistical analysis of the data stream is used to identify the encoding types of the Internet packets. Six encoding methods including western language (ASCII) and the East Asian languages (CJK) can be identified. Results show the high accuracy of the model.