{'fraction_non_alphanumeric': 0.019659239842726082, 'fraction_numerical': 0.005242463958060288, 'mean_word_length': 5.211382113821138, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Improved Data Discretization Algorithm based on Rough Sets Theory

As the key to big data, extracting the potential and core value from a large volume of data is very important. In this study, we propose a discrete data processing algorithm, known as the improved class-attribute contingency coefficient (ICACC) algorithm, based on the rough sets theory and class attribute strain coefficient standard. The proposed ICACC can reduce the data distortion rate effectively by selecting correct discrete points as well as adding the constraint of data inconsistency rate. By testing with the C4.5 algorithm, the proposed ICACC represents a nearly 10 % improvement in terms of the recognition rate and the calculation accuracy, compared with traditional algorithms.