{'fraction_non_alphanumeric': 0.042694147957305854, 'fraction_numerical': 0.012145748987854251, 'mean_word_length': 4.527462946817786, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Improved Measure of Musical Noise Based on Spectral Kurtosis

Audio processing methods operating on a time-frequency representation of the signal can introduce unpleasant sounding artifacts known as musical noise. These artifacts are observed in the context of audio coding, speech enhancement, and source separation. The change in kurtosis of the power spectrum introduced during the processing was shown to correlate with the human perception of musical noise in the context of speech enhancement, leading to the proposal of measures based on it. These baseline measures are here shown to correlate with human perception only in a limited manner. As ground truth for the human perception, the results from two listening tests are considered: one involving audio coding and one involving source separation. Simple but effective perceptually motivated improvements are proposed and the resulting new measure is shown to clearly outperform the baselines in terms of correlation with the results of both listening tests. Moreover, with respect to the listening test on musical noise in audio coding, the exhibited correlation is nearly as good as the one exhibited by the Artifact-related Perceptual Score (APS), which was found to be the best objective measure for this task. The APS is however computationally very expensive. The proposed measure is easily computed, requiring only a fraction of the computational cost of the APS.


INTRODUCTION
Musical noise (sometimes also referred to as musical artifacts, musical tones, or birdies) is an audio distortion caused by isolated holes and/or peaks in the power spectrum of the signal. These can be introduced by audio processing methods operating on a timefrequency representation of the signal, e.g., audio coding, speech enhancement, or source separation. Musical noise is an annoying distortion for the listener, who perceives it as unnatural and warbling fluctuations [1]. Countermeasures and improved methods were developed (e.g., [2,3]) as well as objective measures trying to predict the perceived audio quality degradation, e.g., [4].
In [5], the Artifact-related Perceptual Score (APS) from the Perceptual Evaluation methods for Audio Source Separation (PEASS) toolbox [6] was shown to correlate well with the perception of musical noise (Pearson's linear correlation r = 0.95). However, the calculation of the APS is computationally very expensive. This can be problematic, e.g., if the measure is used in an iterative approach which adjusts the parameters of an algorithm so to maximize the quality of each output signal.
Measures based on the spectral kurtosis (Sec. 2) constitute a computationally lighter alternative to APS. Spectral kurtosis and human perception of musical noise were shown to be correlated in [7,8]. However, in [5] only a limited correlation was found for a previously proposed kurtosis-based measure.
A more in-depth correlation analysis is carried out in this paper (Sec. 4). Simple but effective perceptually motivated improvements are proposed (Sec. 3). The resulting measure is shown to outperform the previously proposed kurtosis-based measures in terms of correlation with perceptual scores. In some cases, the new measure performs as well as the APS at a fraction of the computational cost.

Kurtosis: Definition and General Interpretation
In probability theory, the kurtosis ΨX of a random variable X is defined as the fourth standardized moment: where E {·} is the expectation operator, µn is the n-th order central moment of X (e.g., µ2 is the variance of X), and µ is the mean of X, i.e., µ = E {X}. ΨX is a variance-normalized measure of the weight of the tail(s) of the distribution of X, i.e., a measure of the tendency towards generating outliers. ΨX does not say anything about the shape or the weight of the peak of the distribution of X [9].

Spectral Kurtosis and Musical Noise
In the context of noise reduction for speech enhancement, it was observed that the perception of the introduced musical noise is correlated with the change in kurtosis between the input and the output noise power spectra [7]. It is important to note that the kurtosis change is considered and not the absolute value of the kurtosis itself. Also, only the noise signals are considered. An ideal speech enhancement algorithm would increase the spectral kurtosis while speech is active and leave it unchanged while only noise is present. Hence, it was proposed to quantify the change in spectral kurtosis in the noise signals by means of the log-kurtosis ratio ∆Ψ [7]: where N in and N out are the power spectra of the noise signal before and after processing respectively. In [10], conditions based on ∆Ψ were formulated for a musicalnoise-free processing and used to derive the optimal parameters for an iterative spectral subtraction algorithm.
In [7] and [10] as well as in other works from the same group of authors (e.g., [11,12,13]), speech and noise power spectra are assumed to be gamma-distributed. Complete knowledge of the internal variables of the processing is also assumed and ∆Ψ is analytically derived for each case.

Black-Box Approach
In [8] it is proposed to use the statistical counterpart to ΨX , i.e., to replace the population moments with the sample moments so to obtain an instantaneous kurtosis kurtX ( ): where X is a generic power spectrum, X(k, ) is its k-th frequency bin at the -th time frame, K is the number of bins, and X(k, ) is the instantaneous sample mean of X, i.e., X(k, ) = 1 K K k=1 X(k, ). Hence, kurtNin( ) and kurtNout( ) are obtained by replacing X with N in and N out in Eq. 3. The average over all frames is thus computed, obtaining kurtNin and kurtNout. The black-box log-kurtosis ratio ∆kurt can finally be computed [8]: This measure is considered a black box because it simply needs the signal before and after processing. No assumptions on the distribution of the signal power spectra are made, contrary to [7,10,11,12,13]. Moreover, in [14] it is proposed to replace Eq. 3 by a weighted instantaneous spectral kurtosis kurt W X ( ): where Xα(k, ) = αX (k)X(k, ), and αX (k) represents the signal-and-frequency-dependent weights computed as , where L is the total number of time frames. The resulting weighted log-kurtosis ratio ∆kurt W (obtained by replacing kurtX with kurt W X in Eq. 4) is reported to correlate with perceptual scores with Pearson's linear correlation r = 0.95 [14]. Yet this is evaluated using only 12 scores spanning the entire quality range, i.e., from intolerably audible musical noise to inaudible musical noise. A more extensive correlation analysis is carried out in Sec. 4. The ∆kurt W measure is also recommended in [15].

THE PROPOSED NEW MEASURE
This section proposes a new kurtosis-based measure of musical noise. Following a perceptually motivated pre-processing, modified equations for the calculation of the log-kurtosis ratio are proposed. The pre-processing comprises conversion to A-weighted decibels (dBA), limitation of the lower values, and multi-band analysis. Fig. 1 supports the explanation by visualizing an example consisting of one audio excerpt of a harp arpeggio. The analysis takes place in the short-time Fourier-transform domain using a sine window of 1024 samples, 50% overlap at sampling rate 48 kHz, and discrete Fourier transform length of double the window length.
The spectrogram of the unprocessed signal N in is depicted in Fig. 1(a). The processing under test randomly quantizes to zero a fixed percentage of frequency bins. The output spectrogram N out is depicted in Fig. 1(b). The introduced holes (and peaks) result in clearly perceivable musical noise. However, Fig. 1(c) shows that ∆kurt (Eq. 4) is zero, i.e., the previously proposed ∆kurt fails to detect the musical noise. This happens because the calculation of the spectral kurtosis is dominated by the large number of bins with zero or negligible magnitude. In fact, every bin is identically treated by the previously proposed measures and no aspect of the human auditory system is considered. The proposed pre-processing (Sec. 3.1) targets this problem by focusing on perceptually relevant portions of the spectrogram where differences are evident.

Perceptually Motivated Pre-Processing
It is first proposed to convert the spectral power values to dBA, i.e., the generic power spectrum X(k, ) is converted to X dBA (k, ). This has a dual effect. The logarithmic compression better represents the dynamic range of the human ear, while the A-weighting approximates our frequency-dependent sensitivity to sounds. Then, X dBA (k, ) is limited and shifted so to be non-negative ( + ): where thr is a threshold on the dBA power values. The values below thr are supposed to correspond to silence or negligible levels.
In this implementation thr = P dBA − 20, where P dBA is the overall root mean square (RMS) power of X dBA (k, ). The frames for which N + out (k, ) = 0, ∀k are discarded. Hence, the so-obtained N + in (k, ) and N + out (k, ) are analyzed into sub-bands. In this implementation, 3 sub-bands are used, namely a lower band (50, 750] Hz, a central 3-octaves band (0.75, 6] kHz, and a higher band (6,16] kHz. Frequency outside the range (50 Hz, 16 kHz] are hardly audible by humans and they are not considered. Finer tuning of these bands is left for future works. In each sub-band B, kurt B N in ( ) and kurt B N out ( ) are computed by substituting X(k, ) in Eq. 3 respectively by N + in (k, ) and N + out (k, ) with k ∈ B. Moreover, sub-band energy-weights w B ( ) are computed as follows: where KB is the number of frequency bins in sub-band B. It is thus proposed to turn the attention to the sub-band absolute instantaneous log-kurtosis ratio, defined as: It is worth noting that the previous works (Sec. 2.3) use the time average to compute kurtNout and kurtNin, while here ∆kurt B ( ) is still time-dependent, allowing for an energyweighted temporal mean, as explained in the following.

Perceptually Improved (PI) Log-Kurtosis Ratio
The proposed final measure is thus computed as the energyweighted mean of ∆kurt B ( ): where W = L =1 w B=b ( ), and B =b indicates that one band is selected. The selected band is the one for which L =1 w B ( )∆kurt B ( ) is maximum. This is the band for which the changes in kurtosis are more evident, taking into consideration also the sub-band energy. The band selection aims to resemble the quality perception of a listener, which would be dominated by the worst sounding sub-band. For the example in Fig. 1, this sub-band is B =2, i.e., (0.75, 6] kHz, see Fig. 1(d) and (e). Fig. 1(f) depicts the resulting ∆kurt B ( ) and ∆kurt P I .
In this implementation, ∆kurt B ( ) is limited to 0.5 and ∆kurt P I is linearly rescaled to the range [0, 100] in the following evaluation (but not in Fig. 1).

EVALUATION
This section evaluates the performance of the newly proposed ∆kurt P I and compares it with the baseline kurtosis-based measures ∆kurt and ∆kurt W as well as with the APS. The response to gradually introduced musical noise is studied in Sec. 4.1, while a correlation analysis with perceptual scores is carried out in Sec. 4.2.

Controlled Artificial Distortion
As a first performance evaluation without the need for perceptual data, musical noise is artificially introduced in a gradual and controlled manner. In order to do so, a controlled percentage of randomly selected bins of the spectrogram are set to zero. The percentage ranges from 0% to 99.8%. It is expected that the tested measures exhibit three properties with respect to the distortion control parameter, i.e., monotonicity, low inter-item deviation, and high range spanning. The response score ρ measures to what extend  Figure 2: Responses to gradually introduced musical noise. The distortion control parameter is the percentage (from 0 to 99.8%) of spectrum bins set to zero. The response score ρ [16] is also reported.
these properties are met [16]. In order to be able to calculate ρ, ∆kurt is limited between 0 and 1.4, and ∆kurt W between 0 and 2.2, where the upper limits correspond to the highest scores obtained in this test. The outputs are then linearly rescaled to the range [0, 100]. The test signals are 11 mixtures of stereo background signals and speech recordings (female and male speakers) panned to the center. Each signal is 10 seconds long. The backgrounds comprise music, environmental recordings, and direct signals. Fig. 2 depicts the response of the measures to the introduced musical noise via the mean over all test signals (solid lines) plus/minus the standard deviation (dotted lines). It can be observed that ∆kurt P I outperforms the other kurtosis-based measures, especially in terms of monotonicity and inter-item deviation.

Correlation Analysis with Perceptual Scores
The perceptual scores from the 2 listening tests described in [17] and [6] are here used as ground truth data to evaluate the correlation of the considered measures with the human perception of musical noise. Figs. 3-4 plot the average perceptual scores against the output of the measures. Pearson's correlation r and Kendall's rank correlation t are also reported. The following subsections introduce the perceptual scores and comment on the correlation of the considered measures with these scores.

Listening Test on Audio Coding
In [17], extensive subjective data was gathered; 16 subjects assessed the quality of signals that were distorted in a controlled fashion with different monaural coding artifacts. The listening test was designed so that a process simulating sub-optimal audio coding distorted 8 test items at 5 different coarse quality levels. The perceptual scores are here used as perceptual reference scores, as also done in [5], where the performance of 11 well-known objective evaluation tools was studied. The APS was found to be the measure with the highest correlation with the scores regarding musical noise. In [5], ∆kurt was limited to zero. This is herein referred to as ∆kurt lim.
Ideally, audio coding does not introduce any perceivable change. Hence, for this application, the entire input and output signals can be considered as N in and N out, without the need for selecting the noise components only.
As it can be observed in Fig. 3, ∆kurt P I clearly correlates with the perceptual scores (|r| = 0.95 and |t| = 0.81), nearly as well as the APS. The APS is however computationally very expensive.
The PEASS Matlab implementation [6] takes about 20 seconds for computing the APS of 5 seconds of audio on Matlab 2016 using a quad-core processor Intel i7-3770 at 3.40 GHz without any other competing process running. Under the same conditions, the author's kurt PI mr=0.95,at=0.81I Figure 3: Correlation analysis with perceptual scores from the listening test on musical noise in audio coding [17]. Different symbols are used for the different quality levels, while the dashed lines connect the data points corresponding to the same items in the different quality levels. The x-axis corresponds to average perceptual score, while the y-axis reports the output from the measure mentioned in the title of each plot, where Pearson's linear correlation coefficient r and Kendall's rank correlation coefficient t are also given. The reference signals (scoring 100 points) depicted with black squares are not considered in the calculation of r and t, as they bias the coefficients towards higher values. : Correlation analysis with perceptual scores from the listening test on musical noise in source separation, i.e., the training listening test for the APS [6]. Different symbols are used for the different source separation algorithms. The x-axis corresponds to average perceptual score, while the y-axis reports the output from the measure mentioned in the title of each plot. The references are depicted with black squares, but they are not considered in the calculation of r and t.
non-optimized Matlab implementation of ∆kurt P I requires about 0.1 seconds, i.e., it has two orders of magnitude lower computational complexity than APS. Moreover, the proposed ∆kurt P I clearly outperforms the baselines based on spectral kurtosis, i.e., ∆kurt, ∆kurt lim, and ∆kurt W , which exhibit from low to no correlation with the perceptual scores. ∆kurt P I outperforms also all the other state-of-the-art tools considered in [5] for this task.

Listening Test on Source Separation
The PEASS toolbox (and so the APS) was trained on publicly available results of listening tests focusing on different quality aspects [6]. In one of these, 23 listeners (3 of them were excluded in a screening phase) rated the quality of 10 signals processed by 4 source separation algorithms in terms of absence of additional artificial noise. The results from this test are here used as perceptual reference scores. The content of the signals is very heterogeneous, e.g., multiple overlapping speakers or musical ensembles with or without singing voice. The items have different sampling frequencies, i.e., 16 or 44.1 kHz (resampling to 48 kHz is performed). The results exhibit significant noise, as 10 listeners of the considered 20 failed to detect the hidden reference. For the APS calculation, the reference target and interfering sources are used (i.e., fully intrusive approach). On the other hand, in order to calculate the kurtosisbased measures, simply the activity of the reference target source is used to select the time frames where only the interfering sources are present, i.e., N in and N out. Items 1, 2, and 4 are not considered here, as the target source is always active. For the other items, the time frames available to the kurtosis-based measures are about the 15% of the ones available to the APS. The anchor signals are not considered, as they were created adding musical noise to the target sources only. The noisiness of the perceptual scores, the limited frame number, and the fact that the APS was trained on these scores make the APS a particularly challenging benchmark for this dataset.
As it can be observed in Fig. 4, ∆kurt P I exhibits lower correlation for this test (|r| = 0.71 and |t| = 0.55), but it clearly outperforms the kurtosis-based baselines.

CONCLUSION
This paper proposed a novel measure of musical noise based on spectral kurtosis and on perceptually motivated extensions. The measure was shown to clearly outperform the state-of-the-art kurtosis-based measures both in terms of response score and in terms of correlation with human perception. As perceptual reference scores, the results from two listening tests were considered.
When considering the listening test on which the APS was not trained, the proposed new measure performs nearly as well as the APS, which was found to be the best available measure for this task. Furthermore, the new measure requires two orders of magnitude lower computational complexity than APS.

ACKNOWLEDGMENT
Sincere thanks go to Jouni Paulus, Christian Uhle, Jürgen Herre, Sascha Dick, and Matthew Jefferson for the precious discussions.