{'fraction_non_alphanumeric': 0.00946643717728055, 'fraction_numerical': 0.0, 'mean_word_length': 5.025906735751295, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A fuzzy threshold based unsupervised clustering algorithm for natural data exploration

Traditional clustering methods require the user to determine the number of clusters before we start any data exploration. In fuzzy clustering methods the performance efficiency of the algorithm depends mainly on the initial selection of number of clusters and cluster seeds. The real world data is almost never arranged in clear cut group and the initial selection of cluster count and centroids becomes a tedious task. In this paper we propose a new unsupervised clustering algorithm which works on the principles of fuzzy clustering. The new method we propose is using a modified form of popular fuzzy c-means algorithm for membership calculation. The algorithm begins with two initial cluster centers and forms many clusters based on a threshold value. It uses the fuzzy membership value of a cluster centre in another existing cluster to merge the clusters and finally converges to the optimum number of clusters. The algorithm is tested with the data for Gross National Happiness (GNH) program of Bhutan and found to be highly efficient in segmenting natural data sets.