{'fraction_non_alphanumeric': 0.008839779005524863, 'fraction_numerical': 0.0, 'mean_word_length': 5.565217391304348, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Text clustering based on term weights automatic partition

Text clustering is becoming more and more popular due to the increasing of texts on Web and the requirements in real application. This paper introduces a novel automatic text clustering method, in which the genetic algorithm is first applied to the global optimal and high searching efficient term selection to achieve dimensionality reduction, and then appropriate number of partitions of document set are created according to the different combinations of term weights, and each document partition is clustered into an initial clusters based on dynamic programming technique, and last all initial clusters are clustered using the same method to final text clusters. It also provides analysis and theorem proof that the algorithm can provide higher performance in computational complexity, clustering effect and high dimensional data clustering.