{'fraction_non_alphanumeric': 0.015647226173541962, 'fraction_numerical': 0.0, 'mean_word_length': 5.285714285714286, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A multi-agent system for distributed cluster analysis

One of the approaches used to improve the accuracy and relevancy in information retrieval is cluster analysis. Clustering methods determine relationships among text documents, and allow the determination of similar groups or clusters of documents. These methods are computationally expensive, thereby limiting their use to a relatively small set of documents. This paper describes a multi-agent system to cluster large data sets. This technique is then compared to hierarchical agglomerative clustering using a small set of text data. Results show that the agent-based approach can significantly reduce the time required to cluster large data sets.