{'fraction_non_alphanumeric': 0.019102196752626553, 'fraction_numerical': 0.0038204393505253103, 'mean_word_length': 4.954545454545454, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The optimization of the least-square smoothing of experimental data

A method is developed for the calculation of the square deviation between a smoothed experimental signal and the unknown genuine spectrum. The experimental data must be given (eventually after a mapping) at equally spaced abscissae, and the random noise of the observed signal is supposed to have a zero mean value and a zero correlation coefficient. The method is developed for the linear filters which are obtained by the least-square adjustment of a polynomial of degree n to (2m+1) experimental values. These two filter parameters n and (2m+1) are determined so as to minimize the square deviation of the filtered signal as compared to the inaccessible exact signal. That deviation is shown to be composed of two contributions: the distortion of the exact signal by the smoothing process, and the remaining component of the random noise. These two quantities are also easily calculated. The method is applied to simulated experimental signals and to an actual infrared band.