{'fraction_non_alphanumeric': 0.008492569002123142, 'fraction_numerical': 0.0, 'mean_word_length': 5.883211678832117, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Assessing Information Quality and Distinguishing Feature Subsets for Molecular Classification

A feature reduction scheme is developed in this study. The proposed scheme is a hybrid of unsupervised and supervised methods. The unsupervised process is designed to exclude the irrelevant and useless information before executing feature selection. Fuzzy clustering algorithm will categorize all features into different groups. Then, the supervised process using the target class information will purify the relevant and informative factors from the approved feature clusters. This purifying procedure is particularly suitable for high dimensional datasets with erroneous information. This proposed strategy is to retain the advantage of simplicity and lower selecting time consumption in filter methods while the disadvantage of high computational cost due to the exponential quantity of feature subsets in wrapper methods is completely avoided.