{'fraction_non_alphanumeric': 0.021584004521689983, 'fraction_numerical': 0.00738307192313127, 'mean_word_length': 4.635874975114473, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Autoencoders and Generative Adversarial Networks for Anomaly Detection for Sequences

We introduce synthetic oversampling in anomaly detection for multi-feature sequence datasets based on autoencoders and generative adversarial networks. The first approach considers the use of an autoencoder in conjunction with standard oversampling methods to generate synthetic data that captures the sequential nature of the data. A different model uses generative adversarial networks to generate structure preserving synthetic data for the minority class. We also use generative adversarial networks on the majority class as an outlier detection method for novelty detection. We show that the use of generative adversarial network based synthetic data improves classification model performance on a variety of sequence data sets.


Introduction
Dealing with imbalanced datasets, e.g. anomaly detection, is the crux of many real world classification problems. These problems deal with complex multivariate data such as multi-feature sequence data. Canonical examples can be found in the finance world, for example, questions related to stock market data of several securities or credit card fraud detection often deal with sequence data with many features. Other imbalanced data problems include questions in the medical field such as tumor detection and post surgery prognosis [30]. In each of these problems, false positives are more desirable than false negatives, they require sequential data, and the classes are imbalanced.
Class imbalances in datasets oftentimes lead to increased difficulty in classification problems as many machine learning algorithms assume that the dataset is balanced. There are two general approaches to improve classification accuracy for unbalanced datasets. One method is algorithmic, for example, a modified loss function can be used so that misclassifications of minority labeled data are penalized more heavily than misclassifications of majority labeled data. The other is to decrease data imbalances in the training set either by ensembling the data or by generating synthetic training data to augment the amount of data in the minority set.
This motivates the development of methods to improve classification accuracy on multi-feature sequence data. Given a sequence of T feature vectors, we want to predict labels of the sequence. Oftentimes it is not obvious how to apply methods for unbalanced data to sequence data in a way that takes advantage of the fact that sequential events have the potential to be highly correlated. SMOTE [5] is widely used for oversampling, but does not capture the sequential dimension. Enhanced Structure Preserved Oversampling (ESPO) [3] allows one to generate synthetic data that preserves the sequence structure, however it requires that the feature vector has only a single feature at each of the T time points and that the output label is a scalar. As there is no obvious extension to the case where there are multiple features at each time point and the output is also a sequence of labels, the situations where ESPO can be applied are limited.
We develop methods based on deep learning models for sequences in order to decrease data imbalances of sequence data with an arbitrary number of features. We call each feature vector, x i ∈ R n , an event in the sequence. We primarily focus our methodologies on oversampling. Most methods we propose involve ways to generate synthetic minority examples while preserving the structure imposed by the sequential nature of the data. A proposed approach is training an autoencoder to project a sequence x to a lower dimensional representation and carrying out the SMOTE algorithm on the embedding. Another approach uses generative adversarial networks (GANs) to generate synthetic data. Here, we build a generative model that generates both the feature vectors in a sequence as well as the corresponding labels. Lastly, we consider how GANs can be used for novelty detection type methods where we build a GAN with an autoencoder on the majority class samples. We benchmark these models by comparing them to the baseline sequence-to-sequence model [25]. All of the models are embedded within the standard ensemble approach. On all of our datasets, we observe that only the GAN based synthetic data generation model significantly improves over the baseline model by 15% to 127% depending on the dataset, while the other proposed models perform similarly to the baseline model.
The main contributions are as follows: 1. a method to apply synthetic data generation techniques such as SMOTE and ADASYN to sequence data via an autoencoder; 2. a novel synthetic data generation technique that uses a GAN to generate synthetic data for multifeature sequential data in a way that preserves the structure of sequences for both feature vectors and labels; 3. a new novelty detection method for sequential data that uses a GAN as an outlier detection function.
In the next section, we discuss relevant literature. Section 3 discusses all of the models, while the computational results are presented in Section 4.

Literature Review
Many methods exist for imbalanced data. The majority of these methods are developed for non-sequential data and generally take one of two approaches. The first approach is algorithmic and either involves altering the loss function or performance metric in a way that emphasizes the correct classification of the minority set. The second approach is to decrease the data imbalance either by resampling or by generating synthetic minority data such that the training data is more balanced. The benefit of using algorithmic methods is that they have a straightforward application to sequence data as we can calculate the loss and accuracy the same way for both a vector and a scalar. Methods that are commonly used include a weighted loss function in which the loss of misclassifying minority data is greater than the loss of misclassifying majority data [24]. We implement a weighted loss function in all our models.
In contrast to the algorithmic methods, we can instead consider data level methods that strive to balance the two classes. There have been many different methods that are developed to balance the dataset without generating synthetic minority data. Since these methods alter how the training set is built, applying them to sequence data is straightforward. Both ensembling and data sampling techniques fall under this category. Ensemble methods take the original training set and build subsets of the training set such that the sizes of the minority and majority sets are more balanced [7]. On the other hand, other methods for dataset creation involve over-or under-sampling [14]. Ensemble methods generally outperform over-and under-sampling methods alone so we use ensembles in all our experiments.
Another data level method that can mitigate the class imbalance problem is to generate synthetic minority data. SMOTE [5] is one of the most widely used methods for generating synthetic minority data. For this method, synthetic data is generated via interpolation between nearest neighbors in the minority set. There are many extensions to SMOTE that aim to increase classification performance by sharpening the boundary between the two classes. One such example is Borderline-SMOTE [10], which as the name suggests, generates synthetic data from minority data points whose nearest neighbors include both majority and minority samples so as to strengthen the boundary between the two classes. Another method is ADASYN [11], which explores the composition of the nearest neighbors to determine how many synthetic data points to generate and how to generate them. However, these methods do not have a straightforward extension to sequence data because they build a synthetic feature vector by independently interpolating between the real data points, so the framework cannot capture correlation in time. Structure Preserving Oversampling and ESPO are the only methods, to the best of our knowledge, that exist for dealing with unbalanced sequence data [4,3]. To generate synthetic sequence data, these methods use the covariance structure of the minority data in order to build synthetic minority data that captures the sequential structure. They are developed for single feature sequences and there is not a straightforward extension to data that has multiple features for each event. This is because we cannot calculate the covariance matrix for each feature independently since features may interact with each other in different ways at different events.
Another method for synthetic data generation are GANs [8]. This model pits a generator model, which generates synthetic data, and a discriminator model, which tries to distinguish between real and synthetic data, against each other. By pitting the models against each other, it trains both the generator and discriminator, and once the generator has been trained, we can use it to generate synthetic minority data. While this approach has been applied to image data [29] and sequence data [27], they have not yet been developed to oversample sequence data. GAN based models designed for sequence data have been used for synthetic text generation, but as this architecture is not designed for classification, the sequence class is not considered. These models cannot generate both a sequence and the associated labels.
Both SMOTE and GAN based synthetic data generation techniques have been shown to improve classification performance for certain types of highly imbalanced datasets such as image data or single feature sequences. These models have not yet been developed to sequence data with an arbitrary number of features as even methods developed for generating sequential synthetic data cannot deal with sequence data with more than one feature. GAN based models cannot be directly applied to synthetic minority data generation as the output from the generator is an embedding of the input sequence. So while these methods improve a classifier's performance, unlike the other data-level methods and the algorithmic methods, they have not yet been developed and applied to generic sequence data.
Historically, anomaly detection methods generally use a model such as PCA or SVM to determine which data points are outliers and thus are more likely to be in the minority class [21,12,15,23]. However, novelty detection methods can be improved by the use of more complex outlier detection methods. In deep learning, various LSTM based autoencoder models have been used in novelty detection methods for sequence data so that the outlier detection model can exploit the structure of the data [17,18,19,22]. For the same reason, GANs have also been used for novelty detection methods [26]. However, GAN based novelty detection methods have not yet been tested on sequence data. Therefore, while autoencoders have been used for novelty detection for sequence data and GANs have been used for novelty detection methods on image data, a method that uses a GAN based autoencoder for sequence data has not yet been developed.

Approaches
We assume that we have sequences x = (x 1 , . . . x T ) ∈ X and associated labels y = (y 1 , . . . , y L ) ∈ Y where each x i has n features. We also assume there is a dominant label sequence called majority and all other label sequences are minority. Since we focus on minority sequences, all our synthetic oversampling methods also work with no modification in the presence of multiple majority classes. For the baseline model, we consider a sequence-to-sequence (seq2seq) architecture. This is an encoder-decoder architecture where the entire sequence is represented by an s dimensional hidden vector h 0 T , the encoder hidden state at the final event. We then use this vector, h 0 T , as the input to the decoder model at each event. The model can be written as where f 0 θ E , f 1 θ D are cell functions such as LSTM or GRU [25]. In our experiments, we use a seq2seq model with attention [2] and a weighted loss function where the weights are proportional to class balance as the classification method. We ensemble the data into K ensembles where each ensemble contains a subset of the majority data and all of the minority training data and in inference, we average the predictions from each ensemble.

ADASYN on Autoencoders
In a straightforward application of SMOTE to sequences, we reshape x to a vector and then apply the SMOTE algorithm directly to x. In addition, by reshaping the label y, we can interpolate between the label vectors associated with the samples used to generate the synthetic sample. This creates a fractional valued label that has to be converted to a binary one if the underlying model requires it.
We next discern how to use an autoencoder and SMOTE to generate synthetic data. We first train an autoencoder on minority data. Using the trained autoencoder on the minority data, we obtain h 0 T ∈ R s for each sequence. Once we have embedded the sequence, we can then run the SMOTE algorithm to getĥ 0 T . Next, we can use the decoder half of the autoencoder to liftĥ 0 T back tox. The benefit of this approach is that the encoded minority data captures the structure of the sequence. All that remains is to generate the associated labels for the synthetic data.
An approach is to find a way to use the weights for interpolating between the minority data in the SMOTE algorithm to generate the associated label vector via interpolation. Based on SMOTE for sequence x i , given h 0 T i and h 0 T j obtained from x i and x j , respectively, a synthetic sample T j is one of the neighbors of h 0 T i and w i = (w i 0 , . . . , w i s ) are fixed weights with representing component wise multiplication. Note that this equality does not hold forx, x i , and x j wherex is generated by the decoder with respect to ĥ 0 T syn . We then generate the associated label vector aŝ y = y i +w i (y i − y j ).
wherew i = 1 s ∑ s j=1 w i j . The downside to this approach is that if w i j is a uniformly chosen random number in [0,1], thenw i ∼ 0.5 for s large. Therefore, instead of considering the SMOTE algorithm in conjunction with the autoencoder, we consider the ADASYN algorithm instead. There are two main differences between ADASYN and SMOTE. Instead of choosing weights w i j ∼ U[0, 1], we choose a single random interpolation weight, w i , for each synthetic sample. In addition, the number of synthetic sequences to generate from each sequence in the minority set is adaptively chosen. The label vector,ŷ, associated withx is defined aŝ This method then allows us to apply ADASYN to sequences in a way that should both preserve the structure of the data and generate both sequences and labels.

GAN Based Synthetic Data
We develop a GAN that is capable of generating both sequences, x, and associated label vectors y. As in any GAN model, we must build both a generator and a discriminator and train the models by pitting them against each other. The model that we discuss is based on the improved Wasserstein GAN [9,1]. Recall that in the standard baseline classification model, we use a seq2seq model to get sequences h x and h y of hidden states from sequences x and labels y, respectively. For the generator model, G φ EN 1 ,φ EN 2 (z, x, y) we use LSTM cells to get hidden state sequences h x and h y . We include an additional argument z to initialize the cell state for the generator. For the true data, we set z to 0 and for the fake data we use z ∼ N (0, I). The model is able to distinguish between x and y since x always precedes y in the combined sequence and φ EN 1 and φ EN 2 are parameters corresponding to x and y, respectively. The discriminator model, D φ D 1 ,φ D 2 (h x , h y ) uses a seq2seq model trained on the hidden sequences h x and h y to get a real valued output, c. As in the generator, φ D 1 are parameters corresponding to x and φ D 2 to y. The loss function compares the outputs from the discriminator model for the real and fake data.
We also need a component of the model to decode h x and h y in a meaningful way. Therefore, we have a seq2seq based autoencoder, A φ EN 1 ,φ EN 2 ,φ DE 1 ,φ DE 2 (x, y), that takes as input x and y, creates hidden sequences h x and h y , and then reconstructsx andŷ. The autoencoder shares the encoding part with the generator.
In Figure 1, the GAN with autoencoder structure is sketched out. For model training, we use the loss function where λ and µ are tunable hyperparameters. All expectations are with respect to the minority sequences (x, y).
Generator h x , h y Figure 1: Overview of GAN model. Sequences and labels are used as input to GAN and both the discriminator and decoder use the outputs from the generator model.
During training, we want to prevent the discriminator from learning too quickly so that the generator can learn. We use Adam [13], and set the discriminator learning rate lower than the generator learning rate to prevent the discriminator from learning too quickly. To further slow down discriminator training, we add noise to generator outputs and decrease the noise as model training progresses [6].
During model training, we train the generator, discriminator and autoencoder weights on different batches of data. We first update the weights associated with the generator, φ EN 1 and φ EN 2 , by considering all terms in the loss function. Next, we update the weights associated with the discriminator, φ D 1 and φ D 2 , by including the first three terms of the loss function as the autoencoder loss term does not depend on the discriminator weights. Finally, we update the weights associated with the decoder part of the autoencoder, φ DE 1 and φ DE 2 , using the last term of the loss function. The weights of the encoder part of the autoencoder are shared with the generator, so they are not updated along with the rest of the autoencoder weights.
Once we have trained the generator in conjunction with the discriminator and autoencoder, we can use the generator and the decoder part of the autoencoder to generate synthetic minority data. We generate 3 synthetic samples from each minority sample in the training dataset. This method should improve on the ADASYN with autoencoder model as it allows for the simultaneous generation of both the sequences and associated label vectors.

GAN Novelty Detection and GAN Discriminator Detection
While the previous section trains the GAN model on the minority data, in novelty detection GAN is trained only on the majority data. One approach to novelty detection is to examine the autoencoder reconstruction loss. When computing the autoencoder reconstruction loss on the trained model, we expect the reconstruction loss be higher for the minority class than for the majority class. Similarly, we can examine the discriminator output of the trained model. We expect that the minority class data should be classified as fake data by the discriminator, while the majority class data would be classified as real data. However, since the novelty detection prediction with GAN model on majority data from Figure 1 requires the label vector, y, it needs to be modified.
The model is similar to the model sketched out in Figure 1, except that we use LSTM cells to get the sequence h x of hidden states from sequences x, and the discriminator and autoencoder take as input h x and x, respectively. The generator takes as input noise z and sequence x. The loss function used to train this GAN model is similar to the loss function in (1) and it is trained by using the same logic as the GAN-based synthetic data model. Basically, the model is the same except that the labels y are neglected. We can then use either the autoencoder or the discriminator of this GAN model to classify the majority and minority classes in a novelty detection method. Note that this approach only infers minority/majority classification and not the actual labels y.

Computational Study
We consider three imbalanced datasets 1 . Each of these datasets consists of multi-feature sequence data where the data imbalance is less than 5% (it can be as low as 0.1%). The first dataset is a proprietary medical device dataset where the data comes from medical device outputs. The second dataset we consider is a sentiment analysis dataset that classifies IMDB movie reviews as positive or negative [16]. Though the data is initially balanced, for this paper, we downsample the positive class in order to use it for an anomaly detection task. Lastly, we consider a power consumption dataset 2 where the goal is to predict if voltage consumption changes significantly. A class corresponds to whether the voltage change is considered significant.
For each of the datasets, the data is ensembled into 10 ensembles such that each ensemble contains all of the minority data and a random subset of the majority data. The GAN based oversampling and novelty detection methods are implemented using Tensorflow and the remaining models are implemented using Keras with Tensorflow. We use the Adam optimizer for the GAN based models [6,20], while for the remaining models, we use the Adadelta optimizer [28] in model training. All models are trained on a single GPU card. For each dataset, we tune the number of layers and number of neurons of the baseline model. We use the best performing model as the baseline for comparison.

Medical Device Data
In this dataset, the data is a sequence of readouts from medical devices and the labels indicate if a user error occurs. The sequence length is on average 50 and there are around 50 features. We have on order of 1 million samples and less than 1% of the samples are from the minority class. We make 5 runs, each one with a different seed, and thus each run has different ensemble models. Comparing the results of each of the proposed methods against the baseline in Table 1, we observe that the only method that significantly improves classification accuracy is the GAN-based synthetic data model with p-value = 0.01 based on the t-test. Surprisingly, using the ADASYN Autoencoder generated synthetic data leads to a substantial decrease in the F1-score, suggesting that this synthetic data technique does not capture the structure of the minority data. We also note that the difference in the F1-score between the two novelty detection methods is significant with p-value=2.8e-6 according to the t-test. We observe that the choice of outlier detection is important for novelty detection. To explore how the models trained on the synthetic data improve on the baseline models, we examine the difference between the confusion matrix of predictions on the test set for a model trained with and without the GAN-based synthetic data. In Figure 2, we note that a number of false negatives and false positives in the baseline model are converted to true positives and true negatives, respectively in the model trained on the GAN-based synthetic data.

Run
Examining the classification of true minority and synthetic minority samples in the GAN-based synthetic data training set, we observe that the trained model is better at correctly classifying the synthetic minority samples than the true minority samples which is interesting. For run 0, the F1-score for the true minority training samples is 0.4036 while the F1-score for the synthetic minority training samples is 1. This also reveals that the model overfits since the test F1-score is much lower. This is not surprising for such a heavily imbalanced dataset.

Sentiment
We consider all reviews under 600 words long and front pad reviews so that all samples in our dataset are of length 600. We then use the GoogleNews trained word2vec model to embed the dataset. In order to make this dataset imbalanced, we downsample the positive reviews to create two datasets where the positive reviews comprise 1% and 5% of the training set respectively and then ensemble the training dataset. The resulting dataset is comprised of around 25 thousand samples with 20% in test. Training models on this dataset is computationally expensive because of the sequence length, so we only consider a single run for these experiments.  In Table 2, we compare the results of each of the proposed methods against the baseline. The only method that significantly improves the F1-score is the model trained on the GAN-based synthetic data. We also note that with 5% imbalance, the baseline model performance on the ensembles is high enough that the anomaly detection methods we consider do not improve performance. This suggests that these synthetic data generation techniques are only effective for highly imbalanced datasets.
For this dataset, we conclude that 5% imbalance is an upper bound for which the proposed anomaly detection techniques can be used. However, studying classification of true minority and synthetic minority samples in the GAN-based synthetic data, we notice that the trained model correctly identifies all minority samples in the training set, both true and synthetic. This suggests that the sentiment analysis task is an easier task.
To understand how well the GAN-based synthetic data training set is able to capture the structure of the minority data, we use t-SNE to embed a subset of the true and synthetic minority training data so it can be visualized. In Figure 3, it is clear that the true minority data falls along a line and all but two synthetic minority samples also fall along the same line. As the synthetic samples are staggered along the line, it suggests that for the most part, the synthetic minority data successfully mimics the minority data.  Comparing the results of each of the proposed methods against the baseline in Figure 4, we conclude that the only method that significantly improves the F1-score is the model trained on the GAN-based synthetic data. To test the significance of this improvement, we generate ensembles using 5 different seeds and train a baseline and GAN-based synthetic data model on each run. In the five runs, the average baseline F1-score is 4.45%, the average minority GAN F1-score is 5.10%, and the improvement with the GAN-based synthetic data is significant with p-value=0.018 based on the t-test.
Note that the relative difference in the F1-score between the baseline model and the GAN-based synthetic data model is about 15% and lower than either the Medical Device or Sentiment dataset. As the Power dataset has fewer features than the other two datasets, we observe that the GAN-based synthetic data is better able to capture the data structure for more complex sequences.
On this dataset, we also consider sequences where the associated label vectors are of length 4 by predicting if the voltage change is significant for 4 time periods. As before, sequences are of length 20. We consider a sample as minority if the voltage change is significant in any of the 4 time periods. Approximately 7% of the data is in the minority class. We only consider the GAN-based synthetic data model on this dataset as it is the only model that improves on the baseline in Figure 4. The baseline F1-score is 0.25% and the minority GAN F1-score is 0.59%. Though the imbalance is lower, it is unsurprising that the F1-score is so low as we are making 4 predictions for each sequence. We do not do multiple runs for this dataset as the relative F1-score increase is high. We conclude that the GAN-based synthetic data can be used to improve model performance for datasets with label sequences.

Conclusions
We have presented several techniques for synthetic oversampling in anomaly detection for multi-feature sequence datasets. Models were evaluated on three datasets where it was observed that GAN-based synthetic data generation outperforms all other models on all datasets. We also note that GAN-based synthetic data yielded larger classification F1-score increases over other models for datasets with more features. Furthermore, we provide evidence that the GAN-based synthetic data is capable of capturing the structure of minority data. We also demonstrate that GAN-based synthetic data generation techniques can be applied to datasets with label sequences. Finally, we provide evidence that synthetic oversampling is beneficial for datasets with substantial imbalances (less than 5% in our datasets).