{'fraction_non_alphanumeric': 0.033981359894949235, 'fraction_numerical': 0.012490792044326298, 'mean_word_length': 4.864763335837716, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Robust Audio Anomaly Detection

We propose an outlier robust multivariate time series model which can be used for detecting previously unseen anomalous sounds based on noisy training data. The presented approach doesn't assume the presence of labeled anomalies in the training dataset and uses a novel deep neural network architecture to learn the temporal dynamics of the multivariate time series at multiple resolutions while being robust to contaminations in the training dataset. The temporal dynamics are modeled using recurrent layers augmented with attention mechanism. These recurrent layers are built on top of convolutional layers allowing the network to extract features at multiple resolutions. The output of the network is an outlier robust probability density function modeling the conditional probability of future samples given the time series history. State-of-the-art approaches using other multiresolution architectures are contrasted with our proposed approach. We validate our solution using publicly available machine sound datasets. We demonstrate the effectiveness of our approach in anomaly detection by comparing against several state-of-the-art models.


INTRODUCTION
The task of anomaly detection has many natural applications and has been studied within diverse research areas and application domains such as monitoring financial indicators, machine health, cloud resources, etc. (Ayed et al., 2020). For example, anomaly detection methods are applied to maximize system uptime and to improve operational performance in manufacturing plants by detecting abnormal behaviors (e.g., breakdown) of manufacturing equipment. A concise overview of anomaly detection approaches has been presented in Chandola et al. (2009). A large class of anomaly detection methods assume the existence of only normal data for training. Hence, the main challenge in designing a reliable anomaly detector is to find the best representation of the data which neither generalizes to unseen anomalies resulting in false negatives, nor overfits on existing data and hence, recognize falsely unseen normal data points as anomalies which results in alerting the user too often.
In a realistic setting, it is often challenging to obtain a clean training dataset and it is common to encounter outliers in the training dataset. For example, time series data collected by vibration sensors or microphones in an industrial environment is typically noisy as a result of dynamic interaction of multiple factors. Therefore, a mixture of, generally, non-stationary signals from multiple unknown sources is added to the signal of interest. The limitation of current anomaly detection methods is their lack of ability to deal with contaminated training datasets while capturing temporal dependencies across multiple time steps. In Nalisnick et al. (2019), the limitations of generative models on the anomaly detection / density estimation are discussed, and the necessity of the study on the robust to out-of-distribution inputs are emphasized. The present approach targets this limitation explicitly as it develops a robust multivariate time series anomaly detection model, in which we evaluate the probability of current samples given the time series history using a conditional probability density function learned from the training data. Three key characteristics of our method are: 1) The ability to model temporal dependency in the multivariate time-series data, 2) The ability to track the signal dynamics at multiple resolutions, and 3) Robustness to outliers and noisy training datasets. Our approach is based on an attention mechanism augmented recurrent neural network (RNN) architecture In prior works, anomaly detection methods have been discussed in settings where it is assumed that the input data is either independent and identically distributed or in explicit time series settings, where the temporal dynamic of the time series is explicitly modeled, e.g., using an autoregressive model as it is often the case in traditional change point detection methods (Gustafsson, 2000). In both mentioned cases, approaches to solve the anomaly detection problem are broadly either Generative or Discriminative (Wu et al., 2019).
In the generative approaches, a model is trained to generate samples of the data based on a certain latent representation. If the actual incoming measurement samples and the generated samples differ beyond a tolerable threshold, an anomalous data point is identified. Generative models include the popular autoencoder approaches (Zhou & Paffenroth, 2017;Zong et al., 2018;Gong et al., 2019) and the methods based on the recent neural density estimation (Papamakarios et al., 2017;Germain et al., 2015;Rezende & Mohamed, 2015) such as Yamaguchi et al. (2019); Jadidi et al. (2013); Giri et al. (2020a). Shortcomings of the density estimation based anomaly detectors have been discussed empirically in Nalisnick et al. (2019). They argued that a well-calibrated generative model is not able to assign higher probability to the training data than to some other out-of-distribution data. These shortcomings make the potential power of discriminative approaches apparent.
In the lack of anomalous data, the development of discriminative approaches employs the synthetic generation of anomalous data as perturbations on the normal data, e.g., Giri et al. (2020c). Clearly, the generation of synthetic anomalous data is problem specific and being able to generate relevant anomalous data points is often not a given. In this study, we argue that with generative models state-of-the-art anomaly detection performance can be achieved, if the target distribution is chosen to be outlier robust. Intuitively, a well performing anomaly detector should excel at modelling the boundaries of the normal data distribution and hence, it models the probability distribution of the data "tightly" which can be achieved by outlier robust modeling.
A popular representative of time series methods is the autoregressive model. In this model, the next sample is predicted as a linear function of previous samples while modeling the uncertainty of the model output as allowable innovation of the system (Kailath, 1968). For example, in a multivariate linear prediction setting, often a convolutive model is used where the innovation is modeled as where T , {·} T denotes transpose, L is the prediction context, p is the feature index, and t is the time index. The estimation of the model parameters can be done using least squares approach which implies a Gaussian innovation model. Time series anomaly detection using recurrent architecture has been proposed in Marchi et al. (2015) with application to audio novelty detection where the auditory spectral features are processed by an autoencoder. More recently, a U-Net based architecture with convolutional LSTM layers has been proposed in Zhang et al. (2019). This approach is also temporally multiresolutional as it transforms the input into correlation matrices of multiple time lags. Methods for recurrent density estimation have been proposed in Oliva et al. (2017), where the power of gated-RNNs has been highlighted for this task as such models were able to scan through previously seen dimensions remembering and forgetting information as needed for conditional densities without making any strong Markovian assumptions. Similar architecture for forecasting has been presented in Salinas et al. (2020).

PREDICTIVE ROBUST PARAMETRIC DENSITY MODEL
An overview of the model architecture we are proposing is shown in Fig 1. The model aims at predicting a probability density function (pdf) of a next point given its past using a recurrent architecture where a conditional distribution is modeled as a function of past points using hidden state information h t as follows To be robust to outliers during training, we include a parametric estimation of the conditional probability using a robust probability density function. Hampel (1971) gave a general formulation of distributional robustness in terms of a stability principle. Thereby, the deviations of a certain test statistic should be bounded for bounded deviations of the stochastic model. Consequently, a general approach to obtain an optimum robust estimator is based on the minimax principle (Huber & Ronchetti, 1981). Generally, the minimax approach aims at the least favorable situation for which it suggests the best solution. Hence, in our context, the basic underlying idea for a robust pdf for the next sample is to find a worst case pdf within the given class of models with a certain -deviation. As a suitable definition of the -deviation, a typically introduced model is the -contamination model where p 0 (y) denotes the distribution of the clean data, p outliers (y) the distribution of the outliers in the training data, and is the contamination parameter. As shown in Huber & Ronchetti (1981), considering the maximum asymptotic variance of the estimates for all possible densities within the assumed -deviation yields a powerful approach to optimum outlier robustness. Student t distribution can be less sensitive to outliers than Gaussian distribution by controlling the degree of freedom parameter (Peel & McLachlan, 2000;Svensén & Bishop, 2005;Wong et al., 2009). In this paper, we will show how the Student t mixture model can be seen as a special case of the robust distribution framework presented. The parametric model we assume is where α i denoting the responsibility, c the number of components, and f (y; θ = (µ, Σ, ν)) = Γ((ν + P )/2) Γ(ν/2)ν P/2 π P/2 |Σ| 1/2 · 1 + where µ representing the mean, Σ the covariance matrix, ν the degree of freedom, P the dimension, and Γ(·) is the Gamma function. It is known that the Student t distribution can be expressed as a scale mixture model (Murphy, 2012) with Ga z i | ν 2 , ν 2 being the gamma distribution with the shape and rate parameters being equal ν 2 . Splitting up the integral around the z = 1 offers a representation similar to Eq. (2). For the estimation of the parameters of the assumed model, we use a network inspired by the so-called WaveRNN (Kalchbrenner et al., 2018) and augment this architecture with an attention mechanism as a temporal regularizer as we describe in Appendix A, and a multiresolutional architecture employing convolutional layers described in Appendix B. These two architectural features help the network finding relevant patterns in long sequential data and allow the extraction of global features. The basic core operations of the network at a given resolution are as followš where the W (·) and U (·) matrices are the GRU weights, c t are conditioning parameters, e.g., features extracted as multiple resolutions, u t is the update gate vector, r t is the reset gate vector, • denotes the Hadamard product, the underlined quantities denote a concatenation of the mixture components parameters, Σ lower denotes the components in the lower triangular part of the covariance matrix, Σ diag stands for the diagonal components of the covariance matrix, σ is the sigmoid function, σ + is a scale sigmoid function, and the matrices W ν,{1,2}Σ,µ,α are matrices with block zero matrices selecting the respective output channels of the last layer. Here, we omit the biases for clarity. During the training phase, negative log likelihood (NLL) is adopted as a loss function, which can be written as follows where f (·) is the assumed innovation distribution as discussed above.

EXPERIMENTAL RESULTS
The datasets used for evaluating the presented approach are the same used for DCASE challenge task 2 (DCASE, 2020). The desciptions of the dataset, data preprocessing, and evaluation metrics can be found in Appendix C. We compare the performance of the proposed model with eight baseline models including the DCASE winner (Giri et al., 2020c), the DCASE baseline (Koizumi et al., 2020), two state-of-the-arts anomaly detection models (MSCRED (Zhang et al., 2019) and DAGMM (Zong et al., 2018)), and four traditional anomaly detection models. The traditional models we chose are One-class Support Vector Machine (OC-SVM), Isolation Forest (IF), Autoencoder (AE), and Gaussian Mixture Model (GMM). Each of the baseline models is described in detail in Appendix D. We also consider the four variants of the proposed model for ablation study with respect to multi-resolution, attention mechanism, and Student t mixture model. The four variants are named as follow: (1) RGMM: we don't implement convolutional layers and hence, exclude the multi-resolutional architecture and replace a mixture of multivariate Student t with a mixture of multivariate Gaussian distribution, (2) RGMM-MR: we implement a multiresolutional architecture but replace a mixture of multivariate Student t with a mixture of multivariate Gaussian distribution, (3) RSMM: we don't implement the convolutional layers but use a mixture of multivariate Student t to parameterize the target pdf, and (4) RSMM-MR-W/o Attention: we don't implement the temporal attention mechanism in the proposed model.
The details of the model implementation is presented in Appendix E. The results of the experiment are summarized in Table 1. Additional descriptions including the results on each machine IDs are also reported in Appendix F. RSMM-MR outperforms most of the baseline models, and shows the comparable results to the DCASE winner although the DCASE winner is an ensemble of multiple anomaly detectors. We further show the results that can be obtained by ensembling the presented RSMM-MR with the system described in Giri et al. (2020c) (RSMM-MR with DCASE Winner in Table 1) and show the achieved superior performance of the obtained ensemble. Further, we conducted ablation test with respect to multi-resolution, attention mechanism, and Student t mixture model. The results of the four variants demonstrate the benefit of the individual components in the proposed system. Quantitative ablation analysis is given in Appendix G. Further, an analysis on the robustness to synthetically added Gaussian noise bursts to training data is provided in Appendix H.

A TEMPORAL ATTENTION MECHANISM
To help the network finding relevant pattern in long sequential data, we augment the recurrent layers with the attention mechanism presented in Bahdanau et al. (2014). In this mechanism, while processing a sequence of length L the hidden states of the RNN, h t , are augmented with normalized weights a.k.a. attention vector β = [β 1 , . . . , β L ] which is calculated as follows q l = tanh W l a h t , and β l = softmax W l q q l , where W l a and W l q are learnable weights, q l is a learned embedding for each of the hidden states, and tanh is a hyperbolic tangent activation function. Finally, this attention vector is applied to the states in a sequence as following to obtain a new state vector as an average of the hidden states weighted by β or attention vector B MULTIRESOLUTIONAL ARCHITECTURE The network we propose in this paper is built in a hierarchical manner allowing the extraction of global features (Multiresolution). We achieve this by replicating the original network and feeding it with the inputs after passing them through a temporal filter. As shown in Fig. 2, 1D filters are applied to each time series to extract features over time. The extracted features are fed into GRU as the original network does. The final output layer is then conditioned on the output of these additional networks.

C DATASETS, DATA PREPROCESSING, AND EVALUATION METRICS
The dataset is organized in sound recordings where each recording is a single-channel 10-sec length audio which contains both a target machine's operating sound and environmental noise. The following six types of toy/real machines were used; Toy-car ( There are three machine IDs in each machine type, so the total number of datasets used for training and evaluation is 18. The given task is to detect unknown anomalous sounds that are not observed during the training phase. This can be used to determine whether a condition of target machine is normal or anomalous (e.g., machine failure). For training, only recordings from machines operating under normal conditions are used. More detailed descriptions of the datasets (ToyADMOS and MIMII) and data reprocessing are as follow.
• ToyADMOS (Koizumi et al., 2019). The dataset is provided for audio based machine anomaly detection. The dataset consists of normal operating sounds and anomalous operating sounds. For training, only normal operating sounds are used. The anomalous machine operation sounds were collected by deliberately introducing a fault (e.g., bent shaft, deformation of gear, and excessive tension) in a machine. To simulate actual factory environment, the sounds were randomly mixed with various kinds of environmental noise, which were collected from several places in a real factory. Multivariate time-series are extracted from the raw audio signal either as log Mel or MFCC features using the librosa python library package (McFee et al., 2015); log-Mel feature is used for Fan and Pump and MFCC feature is used for slider and valve. The number of Mel and MFCC is set to 90 (i.e., 90 dimensions), and the features are scaled to a range between -1 and 1.
• MIMII (Purohit et al., 2019). MIMII is the industrial machine sounds dataset that contains sounds of industrial equipment (e.g., valves, pumps, fans, and sliders). The sound of industrial equipment is recorded under normal and anomalous operating conditions. The malfunctions causing the anomalous sound included mechanical unbalanced and looseness in the industrial equipment. Various background noises, which were collected from multiple locations in a factory, were mixed with the recorded sounds to simulate a variety of noisy conditions. We extracted log Mel features from the raw audio signal, and the same preprocessing and scaling methods are used as in ToyADMOS.
For evaluation metrics, we use the same metrics as proposed by the organizing committee of the DCASE challenge (DCASE, 2020), which are Area Under the ROC Curve (AUC) and partial AUC (pAUC). The AUC and pAUC are defined as follows where · is the flooring function and H(x) returns 1 when x > 0 and 0 otherwise, and A θ denotes the model output given the parameters θ. Here, are normal and anomalous test samples, respectively, and have been sorted so that their anomaly scores are in descending order. Here, N − and N + are the number of normal and anomalous test samples, respectively. According to the above formulas, the anomaly scores of normal test samples are used as the threshold for a final decision whether a sample is anomalous or normal.

D BASELINE MODELS
OC-SVM. One-class support vector machine (OC-SVM) (Manevitz & Yousef, 2001) is an unsupervised algorithm learning a decision boundary for anomaly/novelty detection. In this study, we test several kernel types (e.g., linear, RBF, and polynomial), and we simply choose a kernel that renders the best performance. In our experiment, OC-SVM with RBF kernel shows the best performance.
IF. Isolation Forest (IF) (Liu et al., 2008) is a unsupervised anomaly detection algorithm, which explicitly isolates anomalies in the dataset using by means of decision trees. In the method, an ensemble of isolation trees is used to isolate anomalies. To train the model, several parameters (e.g., contamination, the number of trees in ensemble, and maximum number of samples to be drawn to train each base estimator) need to be set, so we simply conduct exhaustive search to find the optimal parameters, and following parameters are adopted: the number of trees = 150, max sample = 256, contamination = 0.05).
AE. Autoencoder (AE) learns a representation of the data, and it's trained to reconstruct an original input. After training, an error between input and output can used as an anomaly score. The method is based on the assumption that anomaly data cannot be effectively reconstructed. We used the same number of layers as done in the DCASE baseline in our AE model, and it is FC(90, 90, GMM. Gaussian Mixture Model (GMM) is a density estimation model to model a complex distribution using multiple Gaussian distributions. Two key parameters of GMM are the number of mixture components and covariance type (full, tied, diagonal, and spherical). We simply perform exhaustive search to find optimal parameters, and ten components with full covariance type are adopted in our study. An expectation-maximization (EM) algorithm is used to fit the model using the training dataset, and log-likelihood is used as an anomaly score during the inference.
MSCRED. (Zhang et al., 2019) Multi-Scale Convolutional Recurrent Encoder-Decoder (MSCRED) reconstructs multi-scale signature matrices, which encode the inter-variable correlations and the temporal dependency. A reconstruction error is used as an anomaly score. The publicly available code is used to implement the model. The size of signature matrices is modified according to the number of dimensions we analyzed. For training parameters, we borrow the wisdom from the original implementation.
DAGMM. (Zong et al., 2018) Deep Autoencoding Gaussian Mixture Model (DAGMM) is a density based model. Sample energy is used as an anomaly score. The publicly available code is used to implement the model. For training parameters, we borrow the wisdom from the original implementation.
DCASE winner. (Giri et al., 2020c) The system is an ensemble of self-supervised models using MobileNetV2, ResnetV2, and a multivariate density estimator GroupMADE. The model is trained for an auxiliary classification task which fits into the self-supervision framework. During the model training, several data augmentation techniques such as synthetic augmentation techniques and spectral warping augmentation were considered, more details can be found in the cited paper.

E IMPLEMENTATION DETAILS
The architecture shown in Fig. 1 presents an overview of the proposed model. We implement the proposed model for the task of anomaly detection. We describe the implementation in four parts; (a) estimating the temporal dynamic of the time series, (b) multiresolutional feature extraction, (c) parameter estimation for a mixture of multivariate distributions, and (d) NLL loss calculation.
In (a), the multivariate time series, x t−1:0 , are passed through the attention augmented recurrent layers. The recurrent architecture used in the model is based on gated recurrent unit (GRU), and the number of hidden layers, sequence length, and hidden size are set to 2, 70 (i.e., 70 frames), and 512. The attention vector is calculated as described in Appendix A.
(b) is a parallel process with (a). In (b), 1D-Convolutional layers (Conv) with kernel length = 10, stride = 3, and padding = 0 are used to extract lower resolution features as described in Appendix B. Each frequency bin (i.e., dimension of the time series is treated as univariate time series) is processed with a 1-D convolutional layer; as shown in Fig. 2, 1D filters are applied to each time series to extract features over time. The output of the convolutional layer is passed through GRU layers similar to (a). As in (a), the outputs of the GRU layer are augmented with an attention mechanism. Outputs from (a) and (b) are concatenated and fed into a fully connected layer for the pdf parameters estimation.
In (c), the parameters of a mixture of multivariate Student-t distribution, which are means vector, covariance matrices, degrees of freedom, and responsibilities, are estimated as described in Section 2. While the means vector and the lower triangular part of covariance matrices are simply estimated using fully connected layers, the specialized activation functions are additionally considered to impose constraints on the diagonal components of covariance matrices, the degrees of freedom, and the responsibilities. For example, the softplus activation function is placed after the fully connected layer to output only positive values for the diagonal components of the covariance matrix. Also, the scaled sigmoid activation function is used to set upper and lower bounds of the degree of freedom (or, to avoid an extremely large degrees of freedom, which would make Student t distribution similar to a Gaussian distribution). In this study, the lower and upper bounds of the degree of freedom are set between 1 and 10. The softmax activation is used to calculate the responsibilities of each mixture components, hence, these values are all positive and sum to 1. The output size of the network for each parameter is a function of the number of components in a mixture (c) and the dimensionality of the data (P ). In our experiments, c is set to 3.
In (d), with the parameters estimated by the model in (c), a mixture model is obtained using Eq. (3), and an NLL loss is calculated for an observed value, x t , through Eq. (4).
The proposed model (RSMM-MR) and the four variants (RGMM, RGMM-MR, RSMM, RSMM-MR-W/o Attention) are trained with the training parameters of epoch = 30, batch size = 128, initialization = uniform with the magnitude of 0.1, optimizer = Adam, and learning rate = 0.00001. L2 regularization with weight decay parameter = 0.001 is chosen. The training and inference are done on a Tesla V100 GPU.
In the implementation of the ensemble of our model with the DCASE winner model (Giri et al., 2020c), we transform the anomaly scores of each model in the ensemble into a standardized scale, before combining them. The standardization transformation for any given model is applied in a permachine manner where we calculate the mean and variance of its anomaly scores over the training data for that machine ID. The anomaly scores are then transformed to have zero mean and unit variance over the training data of that machine ID. Standardized anomaly scores across different models are then combined using mean or max ensembling.

F DETAILED RESULTS OF ANOMALY DETECTION EXPERIMENT
For the anomaly detection task, we use the machine sounds datasets (ToyADMOS and MIMII). We present the average performance of each model in Table 1. Our proposed model, RSMM-MR, demonstrates the superior performance over the most of the baseline models, and shows the comparable results to the DCASE winner. The DCASE winner is an ensemble of MobileNetV2 (Sandler et al., 2018), ResnetV2 (He et al., 2015), and GroupMADE (Giri et al., 2020b, and also most topranked models in the challenge are ensembles of different anomaly detector. Among the traditional anomaly detection models, GMM, which is a density based model, performs the best in all settings in terms of AUC / pAUC, followed by AE, IF, and OC-SVM. While the GMM works well in many datasets, RSMM-MR outperforms in all datasets, especially improving AUC / pAUC by 0.2286 / 0.3417 and 0.4074 / 0.3679 in Slider and Valve, respectively. In our experiment, our approach outperforms MSCRED and DAGMM. In Zong et al. (2018), DAGMM was able to reduce the effect of contamination in trading data, but the anomaly detection accuracy gradually decreased with increasing contamination ratio. DAGMM performed similarly in benchmarks conducted in two recent anomaly detection papers (Goodge et al., 2020;Zhang et al., 2019). Also, ensembling the proposed model with the DCASE winner model further improves the results in all cases.
In Table 2, we report the results on each machine IDs (there are three machine IDs in each machine type). Results of the DCASE winner on each machine IDs were not reported by the authors, hence, we omit this model in Table 2. As it can be observed, our proposed model outperforms the baseline models consistently in most cases. Confirming that the model learns well the representation of the noisy training data (i.e., normal data) without either generalizing to unseen anomalies data nor overfitting on the training dataset. G ABLATION ANALYSIS Figure 3: Ablation analysis-AUC.
We conduct ablation analysis with respect to the different main features of our model to quantify the influence on the overall achieved model. To this end, four variations of the model are implemented,  With the same number of mixture components modelling the innovation process, the average of AUC and pAUC observed in RSMM is higher in all cases than those observed in RGMM, except for Valve. The biggest improvement, 0.0184 (AUC) / 0.0879 (pAUC), is observed in Slider. When incorporating the multiresolutional architecture in the models, i.e., RSMM-MR and RGMM-MR, RSMM-MR shows the better performance than RGMM-MR in all cases. Especially, we observe a big improvement in pAUC; the average pAUC of all machine types is higher around 0.0839 in RSMM-MR than RGMM-MR. This empirical results demonstrate the effectiveness of the Student t mixture model in modelling robust parametric distribution.
To further highlight the effectiveness of the choice of outliers robust distribution to deal with noisy training data, we conduct an additional experiment on ToyCar (ToyADMOS), Slider (MIMII), and Valve (MIMII) where we synthetically introduce noise to the training data. The results are discussed in Appendix H.

G.2 EFFECT OF A MULTIRESOLUTIONAL ARCHITECTURE
We also consider the model with and without multiresolutional architecture described in Section A to demonstrate the effectiveness of extracting global features in the anomaly detection task. Enabling multiresolutional architecture in the model makes the final output conditioned on the output of multiresolutional time-series features. In all cases, the comparisons between RGMM and RGMM-MR or RSMM and RSMM-MR demonstrate that the multiresolutional architecture indeed helps to improve the performance. The averages of AUC and pAUC increase by 0.0348 and 0.0757 after incorporating multiresolutional architecture in RGMM and RSMM, respectively.

G.3 EFFECT OF A TEMPORAL ATTENTION MECHANISM
Lastly, we investigate the effect of the temporal attention mechanism described in Section A. As shown in Table 2 and Figs. 3 and 4, the results of RSMM-MR without the attention mechanism are consistently lower than RSMM-MR. This demonstrates that the recurrent layer augmented with attention mechanism is more effective to capture temporal dynamic in time series data.
H ROBUSTNESS OF THE MODEL TO NOISY TRAINING DATA Figure 5: Results of introducing Gaussian noise bursts to training data To further investigate the robustness of the proposed model, we conduct one more experiment using ToyCar (ToyADMOS), Slider (MIMII), and Valve (MIMII). The effectiveness of an outliers robust distribution modeling the innovation process is highlighted by randomly introducing bursts of zeromean Gaussian noise with variance σ 2 = 5 to 10% of the overall processing frames in spectrogram. The result is summarized in Fig. 5, and it shows the average values of AUC and pAUC of three IDs. As expected, the synthetically induced outliers negatively affect the detection accuracy for both models. However, the performance of our proposed robust model (RSMM-MR) is consistently higher than the non-robust model (RGMM-MR); the average values of AUC and pAUC are higher for Toycar, Slider, and Valve by 0.0366, 0.0986, and 0.2247.