{'fraction_non_alphanumeric': 0.015521064301552107, 'fraction_numerical': 0.0, 'mean_word_length': 5.191780821917808, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Study on data preprocessing algorithm in Web log mining

An important fundamental task for Web log mining is data preprocessing. Now there are some principles about data preprocessing, which has difficulty in application. According to these principles, this paper presents some new rules and algorithms, which can be used to improve the performance of data preprocessing. These rules and algorithms have been proved valid and efficient by experiments.