{'fraction_non_alphanumeric': 0.010141987829614604, 'fraction_numerical': 0.0, 'mean_word_length': 5.861111111111111, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Improved Method of Feature Selection Based on Information Gain

Feature selection is an essential part of text categorization, which can effectively improve classification precision and efficiency. With some drawbacks proposed from traditional IG approach, an optimized approach that takes concentration and distribution into account is proposed for improving IG approach. The experimental results show that the improved IG approach is superior to traditional IG approach in feature selection.