{'fraction_non_alphanumeric': 0.01845018450184502, 'fraction_numerical': 0.0, 'mean_word_length': 5.2, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Denoising of non-stationary signals using optimized Karhunen-Loeve expansion

We show that denoising a non-stationary signal is possible by means of a Karhunen-Loeve (KL) expansion optimized by an entropy criterion. This criterion is used to segment the noisy signal and to choose the most parsimonious KL representation possible for each segment. The entropy of the KL coefficients for different window lengths determines the appropriate number and the lengths of the windows. To find the KL coefficients in each segment, a time-varying autocorrelation matrix is estimated using the evolutionary periodogram. Eigenvalues and eigenvectors needed in the expansion are computed from this matrix. The local eigenvectors are the basis for each segment. An estimate of the evolutionary spectrum of the signal is obtained from the KL expansion. Choosing the KL coefficients corresponding to the most significant eigenvalues and time-windowing are shown to constitute masking in the time-frequency plane. This masking permits the denoising of non-stationary signals corrupted by white noise.