{'fraction_non_alphanumeric': 0.019308943089430895, 'fraction_numerical': 0.0, 'mean_word_length': 5.15625, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Applying feature selective validation (FSV) as an objective function for data optimization

Feature Select Validation (FSV) is a widely used validation method for data comparison. FSV provides a quantitative standard to describe the similarity between two sets of data. In this paper, the application of the FSV technique is extended to data optimization. The raw data obtained from simulations or measurements are often non-ideal for further processing. Several techniques, such as data perturbation, can be used to improve the data quality in certain aspects. However, after modifications the new data could be very different to the original one. Using FSV as an objective function for the optimization process is discussed in this paper, in an example of causality enforcement, to ensure the enforced casual data has the minimum deviations from the original data. The results demonstrate that the proposed approach in this paper is effective for data modification and optimization.