{'fraction_non_alphanumeric': 0.03350253807106599, 'fraction_numerical': 0.0, 'mean_word_length': 5.4868421052631575, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A New Fuzzy Noise-Rejection Data Partitioning Algorithm with Revised Mahalanobis Distance

Fuzzy C-Means (FCM) and hard clustering are the most common tools for data partitioning. However, the presence of noisy observations in the data may cause generation of completely unreliable partitions from these clustering algorithms. Also, application of the Euclidean distance in FCM only produces spherical clusters. In this paper, a new noise-rejection clustering algorithm based on Mahalanobis distance is presented which is able to detect the noise and outlier data and also ellipsoidal clusters. Unlike the traditional FCM, the proposed clustering tool provides much efficient data partitioning capabilities in the presence of noise and outliers. For validation of the proposed model, the model is applied to different noisy data sets. Keywords— Cluster Validity Index (CVI), Fuzzy C-Means (FCM), Possibilistic C-means (PCM), Revised Gustafson-Kessel (GK), Revised Mahalanobis Distance.