{'fraction_non_alphanumeric': 0.017877094972067038, 'fraction_numerical': 0.0, 'mean_word_length': 5.179310344827586, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Categorical Data Clustering using Frequency and Tf-Idf based Cosine Similarity

Clustering is the process of grouping a set of physical objects into classes of similar object. Objects in real world consist of both numerical and categorical data. Categorical data are not analyzed as numerical data due to the lack of inherit ordering. This paper describes Frequency and Tf-Idf Based Categorical Data Clustering (FTCDC) technique based on cosine similarity measure. The FTCDC system consists of four modules, such as data pre-processing, similarity matrix generation, cluster formation and validation. The System architecture of FTCDC is explained and its performance is examined using a simple example scenario. The performance on real world data is measured using accuracy and error rate. The performance of the system much relay on the similarity threshold selected for the clustering process.