{'fraction_non_alphanumeric': 0.02262727844123193, 'fraction_numerical': 0.00251414204902577, 'mean_word_length': 4.940298507462686, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Optimized Background Correction Algorithm in Automated Spectral Analysis Based on Convolution Signals

Two methods for background calculation i.e., those of Taylor and Schutyser and of Van der Plas et al., both based on the use of a zero-area square wave filter, have been critically evaluated. Simulated data, consisting of Gaussian and Voigt profiles superimposed on known background distributions, were used to study the performance of these methods. Statistical noise was introduced on the spectral data with the use of a Gaussian pseudo-random generator. It was found that both methods tend to fail, especially for cases with lines that are interfered with and/or for curved backgrounds. Furthermore, both methods appear to be very sensitive to statistical noise. Therefore, another procedure, also based on the use of a digital filter, is proposed here. In general, it can be stated that this variant performs better than the former ones, especially for weak signals with important statistical fluctuations and for Voigt profiles with parameters a up to 1.0. The main differences between the new procedure and the other ones are: (1) no manual input of the peak position has to be done, which results in a fully automated procedure, and (2) the spectrum is divided into multiplets before reference channels for background calculation are looked up. In our procedure no allowance is given to look for reference channels within the multiplets. It has also been found that correct background corrections can be performed only if a sufficient number of reference channels are available.