{'fraction_non_alphanumeric': 0.02954165893486732, 'fraction_numerical': 0.013508999814436816, 'mean_word_length': 4.911803422553752, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
End-to-End Adversarial White Box Attacks on Music Instrument Classification

Small adversarial perturbations of input data are able to drastically change performance of machine learning systems, thereby challenging the validity of such systems. We present the very first end-to-end adversarial attacks on a music instrument classification system allowing to add perturbations directly to audio waveforms instead of spectrograms. Our attacks are able to reduce the accuracy close to a random baseline while at the same time keeping perturbations almost imperceptible and producing misclassifications to any desired instrument.


INTRODUCTION
Adversarial attacks add marginal and often imperceptible perturbations to input data, thereby drastically changing the performance of state of the art machine learning systems. This phenomenon has been first discovered in image object recognition [1], where high performing systems based on deep learning are highly sensible to imperceptible perturbations of images created by an adversary trying to fool the system.
While adversarial attacks typically do not pose a security threat in the field of Music Information Retrieval (MIR), they allow to question and analyse the validity of MIR systems, as is being done for a music instrument classification system in this paper. For this purpose we evaluate four different adversarial attack approaches on music instrument classification. Starting from an audio waveform, the attacks produce perturbed versions, i.e. adversarial examples, that are misclassified as any instrument we choose, while having a very high signal to noise ratio (SNR) between original signal and added perturbation of up to 44.23dB on average. Contrary to previous results [2] only using spectral representations of audio as image inputs to algorithms from image recognition [1], we also adapt adversarial attacks from speech recognition [3]. This enables us to perform attacks in an end-to-end fashion, instead of on the spectrogram. Another novel aspect of our work is that our data are monophonic instrument or singing voice sounds, not polyphonic songs from different genres, which makes adversarial perturbations easier to hear and hence harder to hide in the original waveform.
In what follows, we start with reviewing related work in section 2, before we introduce our data (section 3) and instrument classification system as well as four end-to-end adversarial attacks in section 4. Thereafter, we compare the different approaches in our experiments (section 5), and discuss and summarize results in sections 6 and 7.

RELATED WORK
Since the vulnerability of neural networks to adversarial examples was discovered [1], numerous different approaches of adversarial attacks have been proposed. These attacks are often differentiated by how much knowledge they use, i.e. white box or black box attacks, and whether they aim to change the classification of an example to an arbitrary (untargeted attack) or specified new class (targeted attack). White box attacks are denoted as such whenever we assume (near) perfect knowledge about a system, and e.g. have access to a model and its parameters. For black box attacks we have very limited knowledge of a system and no access to elements such as model parameters.
A prominent early example of white box untargeted adversarial attacks is the Fast Gradient Sign Method (FGSM) [4]. Instead of trying to decrease the loss of an image with respect to its ground-truth class as is done during training of a classifier, FGSM increases the loss by pushing the image one step into the gradient direction of said loss function. A second untargeted white box method is Projected Gradient Descent applied to the negative loss function (PGDn) [5], where the loss of an input example is maximised iteratively.
Further examples of adversarial attacks from the image domain, including both white and black box attacks, as well as untargeted and targeted methods, are Deepfool [6], the universal adversarial attack [7], the L 0 , L 2 and L ∞ attack [8] and the one-pixel attack [9].
In the field of speech recognition, a white box targeted attack has been proposed by Carlini & Wagner [3], in which the computation of adversarial examples for an endto-end speech-to-text system is formulated as an optimisation problem. This approach is particularly interesting due to its usage of audio and we will refer to it as C&W in the remainder of our paper. Other approaches in the audio domain use psychoacoustic hiding to make adversarial perturbations as imperceptible as possible [10,11].
In work related to ours in this paper, an adversarial attack from the image domain (cf. [8]) is applied to various sound event classifiers with a particular focus on the transferability of adversarial examples between networks [12]. While Figure 1 in section 2 in [12] suggests that the attack is performed end-to-end, this is not clarified any further in the remainder of the work; further work suggests, that there is still a lack of reliable attacks for raw waveform signals [13]. Different research directions in audio are the robustness of adversarial attacks in sound event classification [14], and increasing the robustness of audio classifiers against adversaries, e.g. by detecting adversarial examples [13,15,16].
For the field of MIR, so-called irrelevant random but linear time-invariant audio filtering transformations of the music signal have been used in an untargeted black box attack to both drastically deflate and inflate the performance [17] of genre classification systems. The irrelevance is ascertained with listening tests and real human subjects, with the transformation being audible but not changing the clear impression of a certain musical genre. The same problematic behaviour has also been documented concerning music emotion classification [17] (again using filtering transformations) and rhythm classification [18], where a slight untargeted change in tempo is able to fool the classifier.
Going beyond untargeted filtering approaches, first targeted white box attacks on genre recognition systems have been reported [2], building upon approaches from image object recognition [1]. Essentially magnitude spectral frames computed from audio are treated as images, which however requires non-trivial care to ensure that produced adversarial examples are also valid audio signals.
As for the MIR system we are trying to attack in this paper, instrument classification can be divided [19] into the following tasks, with increasing degree of difficulty: isolated note instrument classification, solo instrument classification, and multi-label classification in polyphonic mixtures. The data used in this paper is a subset derived from a recent audio tagging challenge (see section 3), essentially yielding isolated note and solo instrument data as well as singing voice audio. Our data is therefore an expanded version of isolated note classification, which has largely been solved (see e.g. [20]) and therefore is a good testbed for our adversarial attacks by providing unambiguous class information, clear spectral characteristics per instrument and a difficult setting for hiding adversarial information.

DATA
The data used in this work is a subset of the curated training set provided for Task 2 in the DCASE2019 Challenge [21]. More precisely, out of originally 4970 samples with diverse audio labels, we use all 799 mono audio files which are annotated with one single out of 12 different musical labels: accordion, acoustic guitar, bass drum, bass guitar, electric guitar, female singing, glockenspiel, gong, harmonica, hi-hat, male singing, and marimba / xylophone. The labels are similarly distributed, and occur at least 47 and at most 75 times among the 799 files. In what follows, we use the term instrument to describe both instrumental and singing voice sounds.
All audio files and their manually obtained labels originate from the Freesound Dataset [22]. The labels themselves are weak, i.e. only available at clip-level without starting and ending time information. Audio samples essentially consist of isolated notes or single instrument sounds as well as singing voice audio, with very few of the curated train samples having potentially incomplete labels. In other words, labelled instruments are always present in the audio, but there might be additional instrument sounds which are not represented in the respective annotation.
To reduce computational expenses without significant loss of quality, we re-sample all audio files from 44.1 kHz to 16 kHz. The length of different audio samples varies from 0.3 to 29.29 seconds.

Instrument classification system
Data Pre-Processing: In contrast to previous work on adversaries in MIR, cf. [2], in which adversarial examples are computed for spectrograms, we perform adversarial attacks end-to-end, i.e. on raw waveforms. This avoids having to restrict spectrograms in non-trivial ways to remain valid time-domain signals. To achieve this, we use a differentiable pre-processing module in front of the convolutional neural network (CNN) used for classification. By using torchaudio 1 to realise this first module, we can transform audio into a desired time-frequency representation, and still propagate gradients back to the original waveforms.
To transform raw audio into the time-frequency domain, we first compute spectrograms with a Fast Fourier-Transform (FFT) size of 2048, an equally sized Hannwindow and a hop size of 512. The spectrograms are subsequently mapped to Mel-scale, with 100 Mel-bands between 40 Hz and 8 kHz. Then, the resulting Melspectrograms are transformed to decibel (dB) scale and normalised to have zero mean and a standard deviation of one over the different Mel-bands, across all training data. Whenever we need to lengthen a spectrogram because of network requirements, we perform padding by repeating the spectrogram and inserting the repetitions with equal lengths both before and after the original time-frequency representation. The Architecture: In order to perform instrument classification, we use a CNN with its architecture depicted in Figure 1. The network consists of multiple convolutional layers with 5 × 5 and 3 × 3 convolutions and ReLU activations, followed by batch normalisation [23] and averagepooling layers. Padding values correspond to the strides of convolutional layers. For regularisation, dropout is used. The final layers additionally contain 1 × 1 convolutions and a global pooling layer, which performs global averagepooling over the frequency-, and global max-pooling over the time-dimension of an input. Training and Evaluation Routine: We split the data into a training and validation set containing 599 and 200 audio files respectively. Due to the small size of the dataset, we perform the evaluation of subsequent experiments on validation data. To train the network, we use cross-entropy To allow batch-wise training, we use windowed spectrograms of length 116. For shorter spectrograms, we perform padding as described above; for longer spectrograms, on the other hand, we extract half-overlapping windows of length 116 and choose an arbitrary one for each iteration. When performing validation, we use spectrograms with their original lengths, or padded versions to fulfil minimum length requirements of the network.

Adversarial Attacks
In our experiments, we apply four different adversarial white box attacks to an instrument classification system. Two of these attacks, namely FGSM and PGDn (cf. section 2), are untargeted. The other two are targeted attacks, and both adaptations of the C&W method (cf. section 2). The first version is a slightly modified C&W method, adapted to be suitable for our system. In the second targeted attack, we incorporate a new loss function modelling audio domain knowledge.
To clarify the differences of these approaches, we can look at their optimisation goals which define how an adversarial perturbation is computed. In what follows, ∇ corresponds to a gradient and L net to the cross-entropy loss function, which we also use to train our model. Note that f describes our instrument classification system, including both the pre-processing module as well as the classifier, and x and y denote the original signal and its ground-truth label. The adversarial perturbation that we want to compute is δ, i.e. an adversarial example can be expressed by x + δ. The perturbation δ is always initialised with zeros unless stated otherwise. For iterative methods, η describes the multiplication factor for gradient updates, and k corresponds to the current iteration. Target predictions for targeted attacks are denoted by t. Furthermore, we use a factor α for the targeted methods to control the trade-off between finding adversarial perturbations more easily, and keeping perturbations as small as possible. All gradient updates are performed based on the sign of the gradient. FGSM: The Fast Gradient Sign method [4] is the only single step approach, for which we compute the adversarial perturbation δ by moving a scaled step in the gradient direction in order to increase the loss, i.e.
The parameter λ is a scaling factor controlling the trade-off between difficulty of finding adversarial examples and the quality thereof.
PGDn: Projected Gradient Descent on the negative loss function [5] is an untargeted attack with the purpose of increasing the loss of a sample iteratively. For this attack, we start at a random point close to the original signal, i.e. we initialise the adversarial perturbation by sampling from a uniform distribution U(− , ). After initialisation, we update the perturbation by computing δ k+1 = clip (δ k +η * sign(∇ δ k L net (f (x+δ k ), y))). (2) In each iteration, δ is clipped to stay between − and . C&W: In contrast to the untargeted attacks, in which gradient ascent is used to increase the loss on a ground-truth label to change the prediction of a classifier, we use gradient descent for the targeted methods to minimise the loss with respect to a particular target class t. Instead of minimising the network loss directly, the C&W method optimises a weighted combination of squared L2-norm of the perturbation and a so-called CTC-loss function [3]. As the CTC-loss function is specific to speech-to-text systems, we substitute it with the cross-entropy loss function. Again, C&W clips the perturbation δ in each iteration, leading to The original C&W method uses a re-scaling factor to scale clipped adversarial perturbations in order to stay below certain SNR-thresholds [3]. This factor is decreased as soon as an initial adversarial example is discovered; we omit the re-scaling factor in Eqn (3), as we stop attacks immediately after an initial perturbation is found.
Multi-Scale C&W: The fourth adversarial attack is a different adaptation of C&W [3]. As opposed to using the squared norm to keep perturbations small, we use a multiscale loss [24] in order to make original and adversarial waveform sound as similar as possible. More precisely, we use Note thatx is short for an adversarial example, i.e. x + δ k , and upper-case letters describe a signal transformed to the time-frequency domain. The index i is used to iterate over spectra resulting from different Fourier transformation sizes (2048, 1024, 512, 256, 128, 64; cf. [24]).

Experimental Setup
In what follows, we compare the four end-to-end adversarial attacks introduced in section 4.2. For three of the four methods we use Adam [25] to perform gradient descent and ascent respectively. As FGSM requires only a single gradient update, we apply stochastic gradient ascent instead of using Adam. We also apply a grid-search to find attack-specific parameters, and restrict each iterative method to find an adversarial perturbation within 500 iterations. To further reduce complexity, we sample random target classes different from a sample's original prediction once for targeted attacks, and use these targets in all parameter settings within the grid-search. More information on used parameters for the different methods can be found in section A.1 in the supplementary material. To allow reproducibility of results, all code to run the experiments is available via github 2 .

Results
In Table 1, we compare the four adversarial attacks introduced in section 4.2 based on multiple factors. First, we state the number of adversarial examples found for our 200 validation samples, followed by the average accuracy on the validation set with respect to the ground-truth labels. This accuracy is computed on the adversarial signal of a sample if one was found, or else on the original signal. The last three columns are computed solely on adversarial examples, and depict the average SNR, the median of iterations it takes an adversary to find an adversarial perturbation, and finally the average confidence of our system when classifying an adversarial example. The confidence of a sample here refers to the probability that our system assigns to its predicted class. As the initial perturbation for PGDn and the targets for C&W and Multi-Scale C&W are sampled randomly, we repeat these experiments five times and state mean and standard deviation over all runs. Lines 3 to 6 in Table 1  To put these results into perspective, we take a look at how the accuracy of our system changes if we add random white noise instead of adversarial perturbations to validation samples. When we overlay original signals with white noise of a similar average SNR as the two targeted attacks (42.16dB), the accuracy of our system remains with 0.79 close to the accuracy of 0.835 on clean data.
In Figure 2, three confusion matrices illustrate the accuracy of our system after different attacks. The columns are ground-truth labels, and the rows represent predictions; correct predictions are in the diagonal, and confusions offdiagonal. Each column is normalised to sum to 1, such that a column shows the percentage of correct and incorrect classifications for one ground-truth class. For detailed confusion matrices including numerical values, we refer to section A.2 in the supplementary material. The leftmost image shows the performance on clean data. The majority of predictions is concentrated in the diagonal of the matrix; the classes which are least often classified correctly by our system are 'Marimba and xylophone' and 'Electric guitar'. The remaining two matrices are chosen to depict an untargeted as well as a targeted attack leading to a broad variety of new classifications. The confusion matrix in the middle shows the confusions of our system after a PGDn attack. The untargeted attack leads to diverse new predictions, and appears to have the most difficulties with finding adversarial examples for samples with the groundtruth class 'Male singing', which results in the only no-   When increasing the amount of required adversarial examples to be at least 180, the average SNRs for both C&W and Multi-Scale C&W decrease slightly to 42.98dB and 39.49dB respectively; the average SNR of PGDn, however, drops significantly to 16.47dB. This leads to perceptible adversarial perturbations of PGDn, while the original class remains recognisable. The difference of the SNR of C&W compared to the remaining three methods is statistically significant. Iterations: As the number of iterations it takes for an adversary to find a perturbation for a particular sample strongly varies, iterations are stated as the median or the average ± the standard deviation over medians of different runs. By definition, the single-step FGSM is the adversary requiring the least amount of iterations. The second least iterations are needed by PGDn, taking averages of 15.8 and 2.0 iterations. For the two targeted methods, the number of needed iterations appears to depend on specific parameter settings; in the first case, C&W requires 51.4 iterations as opposed to 71.6 for Multi-Scale C&W. In the second scenario however, Multi-Scale C&W takes less iterations on average with 22.6 iterations in contrast to 66.1 by C&W.
In addition to looking at the number of iterations that different attacks take in various parameter settings, we compare runtime complexities of single iterations with respect to the waveform length N . The gradient ∇ δ L net (f (x + δ), y) is computed in each of the four methods, which is why we consider it to be a given and focus on the complexity of the remaining computations. For FGSM, PGDn and C&W these computations are bounded by N , which is why the attacks are all O(N ). Multi-Scale C&W on the other hand is dominated by the Fourier transformations in the computation of L spec in Eqn (4), and is hence O (N log N ). Confidence: The confidence of our system when classifying adversarial examples tends to decrease with increasing SNRs. More precisely, FGSM achieves the highest confidences with averages of 0.72 and 0.86. Nevertheless, the average confidence of the system does not fall below 0.44.

Using Targeted Attacks to Predict Accordion
Instead of randomly choosing a target class when using the two targeted end-to-end attacks, we can also try to transform any class into a particular target class. To demon- strate this, we use the 'Accordion' class, as it is one of the hardest target classes to reach in prior experiments. For C&W, we find adversarial perturbations for all but 4 samples with the ground-truth classes 'Gong' and 'Male singing'. The adversarial examples have an average SNR of 37.41dB±10.72 and are predicted with an average confidence of 0.44 ± 0.12. As a median, the adversary takes 29 iterations to find a successful adversarial perturbation.
For Multi-Scale C&W, the adversary is successful in finding adversarial examples for all but 7 samples. The adversarial perturbations lead to an average SNR of 37.57dB±10.82, and are classified with an average confidence of 0.44 ± 0.11. The median of required iterations is 33. For more information on the parameters we use for the two methods, we refer to section A.2.6 in the supplementary material.

DISCUSSION
In summary, we see that all four adversarial attacks are able to significantly reduce the accuracy of our instrument classification system. The first untargeted attack, FGSM, is the least complex of the algorithms due to its single-step nature, but also leads to the largest perturbations. PGDn is slightly more complex, and leads to very similar predictions for all adversarial examples whenever high perturbations are allowed. The quality of audio is improved in comparison to the results of FGSM, as we can reach SNRs comparable to the targeted attacks.
The best attack based on the average SNR within our experiments is the targeted C&W method. Despite its comparably high median of required iterations, it still has an asymptotically lower runtime complexity than the targeted Multi-Scale C&W, and reaches average SNRs of above 40dB. Both targeted attacks can be used to transform any class to a particular target prediction, as is demonstrated with the example of the 'Accordion' class.
To put our results into perspective, we look at results from the one already published comparable adversarial attack in MIR [2]. The approach allows setting confidence levels as well as minimal SNRs of adversarial examples. The authors achieve an average SNR of 23.0dB and 15.78dB respectively for two datasets, with a confidence level of 0.9. This setting allows them to reduce the accuracy of genre recognition systems from 0.490 to 0.117 and from 0.632 to 0.413. When decreasing the confidence level to 0.5 for the model with an initial accuracy of 0.632, an average SNR of 11.15dB and a new accuracy of 0.28 is obtained. In a different experiment, in which audio files are transformed to all existing classes, confidence levels of 0.5 and 0.9 respectively lead to average SNRs of 34.5dB and 26.8dB. These results are comparable to ours, as higher SNRs tend to lead to slightly lower average confidence levels. For average confidences between 0.44 and 0.55, we achieve average SNRs between 39.49dB and 44.23dB. Also, we decrease our clean accuracy of 0.835 to accuracies never higher as 0.201. Therefore, while our endto-end methods do not support setting confidence level or SNR beforehand, performing a search over different parameter settings allows us to find examples with slightly higher average SNR and much more reduced accuracy.
The existence of adversarial examples suggests that "being able to correctly label the test data does not imply that our models truly understand the tasks we have asked them to perform" [4] and that impressive results of performance at almost human level might not use musical knowledge at all [17,26]. This directly brings us to the question of validity, i.e. whether our instrument classification experiment is actually measuring what we intended to measure (see [27,28]). Our real intention is to produce algorithms classifying instrument sounds in general, not just for 799 instrument sounds in our data base, which of course extends the target population to all instrument sounds including those with perturbations. If such small perturbations, clearly not changing the perceived instrument characteristics, have such a large impact on the classification system, it must rely on a confounding factor exposing a problem of internal validity, with no causal relation between type of instrument represented in the audio and instrument label returned by the classifier. It is also a problem of external validity, with observed results not generalizing from the original sound material to slightly changed versions thereof. After we have successfully exposed these validity and reliability problems in this paper, our future work will aim at explaining what the confounding factors are and, based on these insights, design a music instrument classification system that is robust against adversarial attacks.

CONCLUSION
In this work, we applied four different white-box adversarial attacks to an instrument classification system. Going beyond previous attacks, we computed adversarial examples directly for the waveform in an end-to-end fashion instead of starting from the spectrogram. The four adversaries were compared with respect to various factors such as runtime complexity, number of adversarial examples that can be found and perceptibility of the adversarial perturbations based on the SNR and listening examples. The successful adversarial attacks were able to decisively reduce accuracy of the classification system with adversarial perturbations of waveforms being almost imperceptible. This paper therefore is a contribution to the growing body of work that questions the validity of MIR results by exposing them to adversarial attacks, thereby gaining deeper understanding of their mode of operation.

ACKNOWLEDGMENTS
This work is supported by the Austrian National Science Foundation (FWF, P 31988).