{'fraction_non_alphanumeric': 0.011077542799597181, 'fraction_numerical': 0.0, 'mean_word_length': 4.812865497076023, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A new approach of gross errors detection for soft sensing data based on cluster analysis

Modeling data plays a very important role in the process of establishing an accurate soft sensing model. Gross errors detection for modeling data could ensure the good quality of modeling data, and then ensure the good performance of soft sensor model. In this paper, a new gross errors detection method based on cluster analysis is proposed. Unlike the traditional methods, the new method does not rely on the mechanism model. And the new method is suitable to the characters of soft sensor better. A new cluster algorithm is presented to detect the gross errors of modeling data based on the special characters of soft sensor. The new clustering algorithm detects the gross errors by analyzing the Euclidean distance between the data points and the center of data set. The experiments demonstrate that the new detection approach based on new clustering method could detect the gross error effectively.