{'fraction_non_alphanumeric': 0.026418786692759294, 'fraction_numerical': 0.006523157208088715, 'mean_word_length': 4.973376623376623, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Piano Timbre Development Analysis using Machine Learning

A data set of recorded single played tones of a concert grand piano is investigated using Machine Learning (ML) on psychoacoustic timbre features. The examined instrument has been recorded at two stages: firstly right after manufacture and secondly after being played in a concert hall for one year. A previous study [Plath2019] revealed that listeners clearly distinguished both stages but no clear correlation with acoustics, signal processing tools or verbalizations of perceived differences could be found. Using a Self-Organizing Map (SOM), training single as well as double feature sets, it can be shown that spectral flux is able to perfectly cluster the two stages. Sound Pressure Level (SPL), roughness, and fractal correlation dimension (as a measure for initial transient chaoticity) are furthermore able to order the keys with respect to high and low notes. Combining spectral flux with the three other features in double-feature training sets maintains stage clustering only for SPL and fractal dimension, showing sub-clusters for both stages. These sub-clusters point to a homogenization of SPL for stage 2 with respect to stage 1 and a pronounced ordering and sub-clustering of key regions with respect to initial transient chaoticity.


Introduction
In a previous study [9], the acoustics, timbre, and perception of a piano have been investigated at two stages, firstly right after manufacture (further denoted as stage 1) and secondly after one year of performance 1 arXiv:2112.03214v2 [q-bio.NC] 16 Dec 2021 in a concert hall (stage 2). In an ABX double-blind comparison test [7] all 88 keys of both stages were presented to subjects. Additionally, subjects were asked to verbalize how they where able to distinguish between stages (regardless of whether they could tell correctly). The text input was cleaned and filtered utilizing natural language processing (NLP) methods and further categorized as affiliated to the domains timbre, pitch, temporal, spatial, and loudness.
Although listeners were significantly able to distinguish between the two stages, data analysis was not able to clearly identify the reason(s). Analyzed acoustical features were driving point mobility, which did not change considerably, the crown bearing, and the action speed. Although the latter two showed differences, like a clear homogenization of the action speed over the year, they could not be associated with perception.
Signal processing of the data sets showed deviations between the two stages: The spectral centroid was considerably lower for the mid-and high-frequency range one year after performance. Still this did not correlate with the distribution of verbalizations in the timbre category. Regarding pitch, the majority of verbalizations was given in frequency-ranges where no measurable pitch differences were present. Also, although verbalizations of temporal differences were reported, no considerable differences of attack time or decay time (RT60) could be observed. Regarding loudness domain features, Sound Pressure Level (SPL) and Interlevel Differences (ILD) decreased in the mid-and high-frequency range one year after playing, still verbalizations were too few to find correlations.
In this paper, a new approach is pursued to distinguish the two piano stages using Music Information Retrieval (MIR) parameters and Machine Learning (ML), namely a Self-Organizing Kohonen Map (SOM).

Method
The apollon and COMSAR frameworks [1,8], developed at the Institute of Systematic Musicology over the last years, have been used within the project of Computational Phonogram Archiving [5,2,3,4]. The system uses the following timbre features to generate a feature vector: spectral centroid, spectral spread, spectral flux, roughness, sharpness, SPL, and fractal correlation dimension. The fractal correlation dimension counts the amount of inharmonic partials and strong amplitude fluctuations which appear during the initial transient of musical sounds [6]. Therefore it is a measure of the chaoticity of initial transients.
All 2×88 single played tones are analyzed with respect to these features in frames of 50 ms over the cause of each sound and the arithmetic mean for each timbre feature is calculated. The aim is to cluster the two piano stages on a two-dimensional Kohonen map. Therefore, different feature combinations are used.
At first, all features are trained by different maps as single features. Therefore the feature vector on each nodal point has only one entry. Secondly, different combinations of feature vectors are used and the resulting trained maps are selected according to clustering of piano stages as well as other sub-clusters within the two piano stages. It turns out that only single and double-features succeed in finding such clusters. Stage 1 keys are red (lowest) to yellow (highest), stage 2 are blue (lowest) to violet (highest). Additionally, the key number is shown next to the dot. Sometimes more than one key is placed on one neuron. The background color shows the u-matrix of the SOM, which is the distance between neighboring neurons. Darker colors imply high similarity for neighboring neurons, lighter colors like yellow mean that the neighboring neurons are very different. Therefore, yellow ridges indicate boundaries between more homogeneous regions displayed in blue.

Single feature SOMs
Spectral flux (upper left) is able to perfectly cluster the two piano stages, where stage 1 is on the upper right and stage 2 on the lower left side of the map. Both stages are clearly separated by a strong ridge in the u-matrix (yellow line). The SOM orders spectral flux as a continuous decrease from a maximum at the lower right corner to a minimum at the upper left corner. Therefore, stage 1 has considerably more flux than stage 2. This might be the main reason for the successful separation of the two stages by listeners found in [9]. Furthermore, stage 2 shows a continues decay of flux from lower keys to higher ones, where stage 1 is not at all so clearly sorted.
SPL (upper right) is not able to cluster the stages, but clearly puts the low keys on the upper left corner where SPL is high and the high keys in the lower right corner where SPL is low. Additionally, next to a main ridge separating the lower right from the upper left corner, two ridges separate a middle region. Therefore, in terms of SPL, four clusters are found, where the two middle clusters sort keys in the medium-frequency range.
Roughness (lower left) clusters lower keys of both stages at the lower left corner and distributes all other keys in the rest of the map. This region of strong roughness, expected from lower keys is strong in the umatrix (yellow region) meaning that additionally to strong roughness, the low keys are also very diverse with respect to roughness.
The map for fractal correlation dimension (lower right) behaves similarly to the one for roughness, pointing to a correlation between the two parameters with respect to piano sounds. Training a SOM with only these two timbre features does not result in a perfect orthonormal relation, but also not in a perfect correlation.
Therefore, spectral flux is assumed to be the main cause of listeners being able to distinguish the two stages aurally. The other timbre parameters go into details which is further investigated using timbre combinations below. Other timbre features did not result in any clear clusters.

Double-feature SOMs
As shown with multi-dimensional scaling (MDS) listening tests over the last decades, timbre perception is multi-dimensional (for a review see [6]). Therefore it is likely that listeners use a combination of timbre features for distinguishing between tones. Almost all MDS investigations resulted in a maximum of three timbre features, where the third one is most often very weakly present, hereinafter combinations of two features are presented. Since spectral flux is the only feature being able to distinguish between stage 1 and 2 and as listeners were asked for performing a difference discrimination, spectral flux was used in all three combinations discussed below. The respective second features used in combination with spectral flux are SPL, roughness, and fractal correlation dimension, as again only these features did result in reasonable clusters as single features (see above).  Another strong ridge can be seen in the lower right side which is considerably stronger for stage 1 than for

Conclusions
Spectral flux clusters the two piano stages perfectly and therefore is assumed to be the main cause of the clear separation of the two stages by listeners found in [9]. Furthermore, stage 2 has a much smoother distribution of SPL and a clearer sub-clustering structure of the chaoticity of initial transients. This is expected    due to the one-year treatment of piano tuners of stage 2 compared to state 1, which seem to have these features as their aim. Assuming at least two timbre parameters to be used by listeners to separate the two stages, two candidates appear from the present analysis, being spectral flux with SPL and spectral flux with fractal correlation dimension.