{'fraction_non_alphanumeric': 0.03387458006718925, 'fraction_numerical': 0.014221724524076149, 'mean_word_length': 4.995636119503189, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Influence of Event Duration on Automatic Wheeze Classification

Patients with respiratory conditions typically exhibit adventitious respiratory sounds, such as wheezes. Wheeze events have variable duration. In this work we studied the influence of event duration on wheeze classification, namely how the creation of the non-wheeze class affected the classifiers' performance. First, we evaluated several classifiers on an open access respiratory sound database, with the best one reaching sensitivity and specificity values of 98% and 95%, respectively. Then, by changing one parameter in the design of the non-wheeze class, i.e., event duration, the best classifier only reached sensitivity and specificity values of 55% and 76%, respectively. These results demonstrate the importance of experimental design on the assessment of wheeze classification algorithms' performance.


I. INTRODUCTION
Respiratory conditions are leading causes of morbidity and mortality worldwide [1] and generate a significant burden for public health systems and society [2]. Early diagnosis and routine monitoring of patients with respiratory conditions are important for timely interventions; thus, considerable research efforts have been focused on these issues in the last couple of decades [3].
Respiratory sounds are a simple, objective, and noninvasive marker to assess patients' respiratory condition [4]. In clinical practice they are commonly assessed with pulmonary auscultation using a stethoscope. The expert clinician is trained to listen to and to recognize pathological findings, such as the presence of adventitious sounds (e.g. crackles, wheezes), commonly in the anterior and posterior chest of the patient [5]. Conventional auscultation is commonly applied but has some drawbacks, such as the inability to provide continuous monitoring and its inter-listener variability. Automated respiratory sound analysis could potentially overcome these limitations.
Respiratory sounds are divided into normal and abnormal categories. Normal respiratory sounds are nonmusical sounds produced from breathing and heard over the trachea and chest wall [6]. They show different acoustic properties, such as duration, pitch, and sound quality depending on subjects' characteristics, subjects' position, respiratory flow, and place of recording [7], [8]. On the other hand, adventitious sounds are abnormal sounds that are superimposed on the normal respiratory sounds [6], [9]. Adventitious sounds can be categorized in two main types, continuous and discontinuous [10]. The nomenclature preconized by the standardization of lung sound nomenclature taskforce [11], i.e., continuous adventitious sounds are referred to as wheezes, while discontinuous adventitious sounds are called crackles, will be followed in this study.
Crackles are explosive, short, discontinuous, and nonmusical adventitious sounds that are attributed to the sudden opening and closing of abnormally closed airways [12]. In contrast, wheezes are musical respiratory sounds usually longer than 80-100 ms and with frequencies ranging from less than 100 Hz to 1000 Hz, with harmonics that occasionally exceed 1000 Hz [13]. Wheezes occur when there is a flow limitation and they can be produced by all mechanisms that reduce airway caliber. Clinically, they can be defined by their frequency (mono-or polyphonic), intensity, number, duration, and position in the respiratory cycle (inspiratory or expiratory), gravity influence, and respiratory maneuvers [12]. Wheezes have been used for diagnostic purposes in several respiratory conditions in children (e.g., bronchiolitis) and in adults (e.g., chronic obstructive pulmonary disease (COPD)) [12].
In this work we propose several machine learning methods for the classification of wheeze events against other types of events, randomly selected. The main objectives of this study were to: a) develop several wheeze classification models (traditional and deep learning approaches) and b) study the influence of the randomly generated events on the classification performance.
The article is organized as follows: in section II, we provide a general overview of several state-of-the-art algorithms that have been used in similar works; in section III, we present the database, as well as showing all the methods used in the different experiments; in section IV, the obtained results are displayed; and, lastly, we discuss the results and give a conclusion in section V.

II. RELATED WORK
With the onset of pattern recognition, several features and machine learning approaches have been proposed in the literature to develop automatic systems for wheeze classification. Pramono et al. [7] identified the most common features and machine learning algorithms employed in the literature. These include Mel-frequency cepstral coefficients (MFCCs) [26], spectral features [27], entropy [28], wavelet coefficients [29], support vector machines (SVM) [17], artificial neural networks (ANN) [16], rule-based models [30] and logistic regression models [18]. Over the years, several authors have reported excellent results on the wheeze classification task. A summary of selected works can be found in Table I. However, one crucial problem of this field has been its reliance on small or private data collections. Besides, public repositories that have been commonly used in the literature (e.g., R.A.L.E. [31]) were designed for teaching, typically include a small number of adventitious respiratory sounds, and usually do not contain environmental noise. Therefore, we chose to perform the evaluation on the Respiratory Sound Dataset (RSD), the largest publicly available respiratory sound database, which is described in section III.

A. Database
The Respiratory Sound Database (RSD) is a publicly available database with 920 audio files containing a total of 5.5 hours of recordings acquired from 126 participants of all ages [32]. The database main characteristics are described in Table II. The database contains audio samples that were collected independently by two research teams in two different countries and it is a challenging database, since the recordings contain several types of noises, background sounds and different sampling frequencies. It contains annotations of 1898 wheezes, which are found in 341 audio files. The training set contains 1173 wheezes distributed among 203 files, while the testing set includes 725 wheezes distributed among 138 files.

B. Random Event Generation
To establish a comparison with other methods that were tested on this database, we mimicked "Experiment 2" of [32] but focusing only on wheeze classification. As detailed in that experiment, we generated random events using a custom script. To simultaneously guarantee variation and reproducibility, the seed for the random number generator changed for each file but was predetermined. In this work, we employed two approaches to generate the random events. Both approaches generated the same number of events, as our goal was to study the impact of event duration on the algorithms' performance. To keep the dataset classes' proportion similar to the original (40% wheezes, 60% other), we established a rule imposing a limit of one event per 5 s, i.e., the script generated at most four random events in a 20 s file, depending on the number of annotated wheezes. A total of 2910 random events were generated, 1781 in the training set and 1129 in the test set. In the first approach, following [32], events with a fixed duration of 150 ms were generated. In the second approach, we visually inspected the distribution of the annotated wheezes' durations and found that a Burr distribution [33] provided a good fit. The Burr distribution used to generate the events had probability density function: (1) with α = 0.2266 , c = 4.1906 and k = 0.3029.
All the random events were then generated with a duration belonging to this distribution, ranging between 100 ms and 2 s.

C. Preprocessing
The audio files in RSD were recorded with different sampling rates. Therefore, we resampled every recording at 4000 Hz, the lowest sampling rate in the database. As the signal of interest was below 2000 Hz [13], this was considered a good resolution for Fourier analysis.

D. Spectrogram
To generate the time-frequency representation of the audio events, the Short-Time Fourier Transform (STFT) was used. The STFT is one of the most used tools in audio analysis and processing since it describes the evolution of the frequency components over time. The STFT representation (F) of a given discrete signal is given by [25]: where ω(i) is a window function centered at the instant n. Since the database events had a wide range of durations, a maximum time for event was defined. Using the typical duration of an expiratory cycle, the maximum size per event was established as 2 seconds, and smaller events were centered and zero-padded. The database also contained annotated events longer than 2 s (77 events). For these cases, only the 2 first seconds were considered, as these bigger annotations were regarded as less precise. This stresses the need for a future review and correction of the database annotations.
Spectrograms with a 128 ms hamming window and 75% of overlap were generated. The spectrograms were normalized between 0 and 1. The spectrogram images used by the CNN had 59 by 257 pixels. In Figure 1, it is possible to observe an example of two representative events.

E. Feature Extraction
In this step, 47 features were extracted from each frame of the spectrogram using the MIR Toolbox [34]. Most of these features were used in [18] for wheeze classification and are commonly used in the field of music information research. Table III provides a small description of all the features. Please note that the selected settings were based on previous research and it was not possible to validate all the frame sizes and other extraction parameters for this study. For each event, five statistics of each feature were calculated: mean, standard deviation, median, minimum value, and maximum value. Therefore, the total number of features fed to the classifiers was 235.

F. Classifiers
To classify the samples as wheezes or other we used several machine learning algorithms: linear discriminant analysis (LDA), Linear SVM (SVMlin), Gaussian SVM (SVMrbf), boosted trees (Boost), and a CNN (using the spectrogram images as input). All the classifiers were trained 10 times with different seeds and their parameters were optimized on a validation set containing 25% of the training set. The models with the best parameters were then applied to the testing set. Bayes optimization was used to optimize the following parameters of each traditional machine learning algorithm: delta and gamma for LDA; box constraint and kernel scale for SVM; method, number of learning cycles (NLearn), and learning rate (LearnRate) for Boost. The best parameters for the best run of each system are listed in Table IV. To develop the CNN model, a base architecture was defined and the parameters of each layer were then optimized using a grid search approach. The optimized parameters were the following ones: Convolution size; Number of convolutional filters; Dropout rate; Pooling size; Size of the fully connected layer. The models were trained with a maximum number of 15 epochs, a mini batch size of 128, Adam optimization (adaptive moment estimation), and 10% of the training dataset was used for validation during the training phase. The best models' architectures for both experiments are described in Table V. A system consisting of all the features and a logistic regression model was used as a baseline for comparison.

G. Evaluation Metrics
To evaluate the algorithms' performance, we used the following measures: P recision(P rec) = T P (T P + F P ) (4)

IV. EVALUATION
In this section, we analyze the performance of the algorithms on two experiments corresponding to the two approaches for the generation of random events. We focus especially on MCC, as this is the only evaluation measure that simultaneously considers TP, TN, FP, and FN. To analyze statistical significance, we performed the right-tailed Wilcoxon sign rank test, for a significance level of α = 1%, against the baseline. To adjust for multiple comparisons, we applied Bonferroni correction to the alpha value.

A. Fixed Durations
Table VII displays the results achieved by all the combinations of classifiers and feature sets on the fixed durations (FD) test set. Figure 2 shows the MCC results with the corresponding error bars.
As mentioned in section II, the best system's wheeze Sens and Spec in [32] were 78.6% and 90.5%, respectively. With Sens and Spec of 79% and 97.9%, respectively, our baseline's performance was on par with the state-of-the-art. In addition, it achieved better results than the other traditional classifiers (MCC of 80.3%). Nevertheless, the CNN achieved even better results, reaching an MCC of 91.8%. Considering the MCC, only the CNN performed significantly better than the baseline.

B. Variable Durations
Table VIII displays the results achieved by all the combinations of classifiers and feature sets on the variable durations (VD) test set. Figure 2 shows the MCC results with the corresponding error bars.
The decline in performance is quite noticeable. The best classifier in this experiment, Boost, reached an MCC of 31.7%. However, none of the optimized algorithms significantly outperformed the baseline (MCC of 30.6%).

C. Variance in performance
The substantial variance in performance between experiments might indicate that the generation of the random events with fixed durations introduced considerable bias.    Figure 3a, all the misclassified wheezes had short durations of less than 325 ms. When classifying wheezes against random events of variable duration, the distribution of FN durations was quite different, encompassing all durations, as shown in Figure 3b. When using the CNN to perform the same classification, the effect of implicitly learning the duration of events was even more noticeable, as can be seen when comparing Figure 3c and Figure 3d. All the misclassified wheezes in the fixed durations test set belonged to the same histogram bin, comprising durations between 125 ms and 175 ms. When considering the variable durations, the CNN model's FN distribution also covered the full range of durations. To improve visualization, all the annotated wheezes longer than 2 s were equalized and assigned to the same histogram bin. Figure 4 illustrates how the same algorithm attributed different classes to the same events depending on how the nonwheeze class was designed. On the FD experiment, all annotated wheezes in this sound file were correctly classified, while on the VD experiment, two out of six annotated wheezes were incorrectly classified. Through visual inspection of the spectrogram alone, it is not possible to understand why the 3 rd and 6 th wheezes were correctly classified while the 4 th and 5 th wheezes were not, as they seem to show similar spectrogram signatures. This figure also reveals potential annotation inaccuracies, as the spectrogram suggests a possible wheeze that was not annotated between 19.4 s and 20 s.

V. DISCUSSION AND CONCLUSION
In this work, we proposed and evaluated several wheeze classification systems, and demonstrated how event duration can have a significant impact on automatic wheeze classification. As the algorithms' performance presented in the previous section show, methods that seem to achieve promising results can fail if we change the way the non-wheeze class is designed. This can happen even if the dataset where the systems are evaluated does not change. It is important to consider how data are used to train, validate, and test a classification algorithm. Our goal was not to question state-of-the-art results in wheeze classification, but to emphasize the critical role of experimental design in the evaluation of the algorithms' performance. As asserted by Sturm [35], when the performance of an artificial system appears to support the claim that the system is addressing a complex human task (e.g., classify sounds as wheezes), the default position (null hypothesis) should be that the system is (a) FD test set (best run -Boost).
(d) VD test set (best run -CNN).  not actually addressing the problem it appears to be solving; in this case, a system that claims to distinguish a wheeze from a random event is implicitly learning an irrelevant characteristic of the dataset, i.e., event duration. To determine if a system is relevant, we need to understand the extent to which the characteristics it is extracting from the signal are confounded with the ground truth, i.e., the expert's annotations. Besides, investigations like this are only possible if algorithm evaluation is performed on open access datasets. Nevertheless, it is important to reiterate the limitations of this database, which may have influenced the performance of the evaluated systems. As pointed out in [32], these include the lack of healthy adult participants and the absence of gold standard annotations, i.e., annotations from multiple annotators. A future update of the database should also check for possible errors.
Wheeze classification is a complex task that is not yet solved, despite the claims made in the literature. It may be particularly hard when algorithms are evaluated on challenging datasets, such as the RSD. Though significant work has been developed to classify wheezes, none has been widely accepted [36]. While CNNs have become state-of-the-art solutions in several tasks [9], a simple CNN with no preprocessing was not enough to tackle this problem. Therefore, accelerating the development of machine learning algorithms is critical to the future of respiratory sounds analysis.