{'fraction_non_alphanumeric': 0.04510026216963084, 'fraction_numerical': 0.00878622546588252, 'mean_word_length': 4.895363408521304, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Regularized Cycle Consistent Generative Adversarial Network for Anomaly Detection

In this paper, we investigate algorithms for anomaly detection. Previous anomaly detection methods focus on modeling the distribution of non-anomalous data provided during training. However, this does not necessarily ensure the correct detection of anomalous data. We propose a new Regularized Cycle Consistent Generative Adversarial Network (RCGAN) in which deep neural networks are adversarially trained to better recognize anomalous samples. This approach is based on leveraging a penalty distribution with a new definition of the loss function and novel use of discriminator networks. It is based on a solid mathematical foundation, and proofs show that our approach has stronger guarantees for detecting anomalous examples compared to the current state-of-the-art. Experimental results on both real-world and synthetic data show that our model leads to significant and consistent improvements on previous anomaly detection benchmarks. Notably, RCGAN improves on the state-of-the-art on the KDDCUP, Arrhythmia, Thyroid, Musk and CIFAR10 datasets.


Introduction
Anomaly detection refers to the task of identifying anomalous observations that deviate from what are believed to be normal data. It has been an important and active research area in many domains, such as medical diagnosis [21], cyber intrusion detection [3] and robotics [14]. Emerging deep learning models [8] with extraordinary capacity to estimate the complex distributions in high-dimensional data provide new approaches for anomaly detection. Efforts have been made to address anomaly detection by deep neural networks, for example energy-based model [24] and deep Gaussian mixture model [25].
Recently proposed bi-directional Generative Adversarial Networks (GANs), including Adversarial Learned Inference (ALI) [6] and ALI with Conditional Entropy (ALICE) [9], allow for highquality mapping back from data to latent variable space via an encoder network. Adversarially Learned Anomaly Detection (ALAD) in [23], which is built upon ALICE, leverages reconstruction of data from GAN for anomaly detection. Despite inspiring progress in deep anomaly detection, most of the previous work focuses on density estimation based primarily on normal data, i.e. the generation of normal In this paper, we propose a theoretically grounded algorithm based on GAN with special modifications to the loss function and discriminator networks to bias both the generator and the discriminator towards the normal manifold. We introduce a penalty distribution t(x) w.r.t. the normal data distribution q(x) in the adversarial training such that the generation from the latent space is biased towards normal manifold and discriminator is trained adversarially to assign higher probability to normal data. The penalty distribution is chosen to be a random noise distribution (e.g., Gaussian or uniform distribution) and the motivation will be explained in section 4. Mathematical proofs show that the introduction of the penalty distribution results in a more consistent detection of anomalous data (avoiding false positive). Results on synthetic data also demonstrate that RC-GAN yields more faithful discriminators for anomaly detection than previous GAN-based models.
We evaluate our approach on real-world datasets including KD-DCUP (network intrusion), Arrhythmia, Thyroid (medical diagnosis), Musk (molecular chemistry) and CIFAR10 (vision). On all these datasets, RCGAN outperforms all other baseline models, achieving the new state-of-the-art.
In summary, our key contributions are two-fold: • The introduction of the penalty distribution t(x) to GAN-based framework for anomaly detection to bias the generator and the discriminator towards the normal manifold. • Mathematical proofs show that RCGAN enforces a large reconstruction error on anomalous data and encourages accurate reconstruction for normal data, providing a theoretical guarantee for reliable anomaly detection.

Related Work
Also known as novelty detection and outlier detection, anomaly detection has been extensively studied in literature. Previous methods can be roughly categorized into two types, representation learning and generative model. Representation learning methods address anomaly detection by extracting common features or learning a data mapping from normal data. One-Class Support Vector Machines (OC-SVM) [22] finds a maximum margin hyperplane such that mapped normal data are separated from the origin. [18] optimizes a hypershpere to enclose the network representations of the normal data. [10] leverages temperature scaling and perturbations upon a a pre-trained neural network for image anomaly detection. [7] develops an approach for vision anomaly detection by training a deep classifier on geometrictransformed normal images. The classifier essentially provides feature detectors with softmax activation statistics that can be used to compute anomaly scores.
Generative models mostly try to learn the reconstruction of data and detect anomaly through reconstruction profiles. [2] uses variational autoencoder and computes the reconstruction probability through Monte-Carlo sampling. DSEBM [24] learns an energy-based model to map each example to an energy score. DAGMM [25] models Gaussian mixture from normal data via an autoencoder network. In [15] researchers use a probabilistic approach to compute how likely one sample is anomalous via adversarial autoencoders. In [19] authors introduce distorted normal data to the training of autoencoders, however, theoretical analysis are lacking why the method works. Recently, Generative Adversarial Networks have been explored for anomaly detection. ADGAN [21] is proposed to identify disease markers on tomography images of the retina. Images are mapped back to the latent variable space by a recursive backpropagation process. ALAD [23] adopts a bi-directional GAN framework, and data are mapped back to the latent space by the encoder network. [13] uses DCGAN with a multi-hypotheses autoencoder, and the anomaly score is based on local neighborhood.
Among all generative methods, our work is mostly related to ALAD. In contrast to ALAD, RCGAN utilizes samples from an a priori chosen random noise distribution as adversarial data during training and enables discriminators to better recognize anomalies, as suggested by our theoretical analysis and experimental evaluations.

Preliminaries
In this section we briefly introduce the anomaly detection problem from a statistical angle. Then we go through closely related GAN frameworks and their applications for anomaly detection. Finally we motivate why the application of penalty distribution in the training of GANs is essential.

Anomaly Detection from a Statistical Perspective
The anomaly detection problem can be formulated as follows. We consider that the "normal" data is defined by a probability density function q(x) ("normal" does not refer to Gaussian distribution in our paper unless mentioned). In unsupervised anomaly detection, during training, we only have access to samples from q(x). The goal is to learn an anomaly score function A(x) such that, during the test phase, anomalous examples are assigned with larger anomaly scores than normal examples.

Generative Adversarial Networks
One approach for anomaly detection is to model the underlying distribution of normal data q(x) based on training normal examples. Generative Adversarial Networks (GANs) [8] can model a distribution using a transformation G(z) from a latent space distribution p(z) to the space x; the generator network G(z) defines the conditional distribution p(x|z). The generator distribution is defined as The GANs framework trains a discriminator network D(x) to distinguish between real data from q(x) and synthetic data generated from p(x). The minmax objective function for GANs is: (1) [8] shows that the optimal generator and discriminator correspond in Eq. (1) to a saddle point such that the generator distribution matches the data distribution p(x) = q(x).
Adversarially Learned Inference (ALI) from [6] introduces an encoder network E(x) and attempts to match the encoder joint distribution q(x, z) = q(x)e(z|x) and the generator joint distribution p(x, z) = p(z)p(x|z), where e(z|x) is parameterized by the encoder network. The same idea is also proposed in [4]. The optimization objective for ALI is defined as: where Dxz is a discriminator network that takes both x and z as input and the output is the probability that x and z are from q(x, z). It follows that the optimum of the encoder, generator and discriminator is a saddle point of Eq. (2) if and only if q(x, z) = p(x, z). Also if a solution of Eq. (2) is achieved, the marginal and joint distributions in (x, z) match.
In order to address the non-identifiability issues in ALI, [9] proposes ALI with Conditional Entropy (ALICE) that adds a second discriminator Dxx network to ALI to distinguish x and its reconstructionx = G(E(x)). An extra term is included in the overall optimization objective, written as: [9] shows that Eq. (3) approximates an upper bound of the conditional entropy H π (x|z) = −E π(x,z) [log π(x|z)], where π(x, z) represents the matched joint distribution π(x, z) q(x, z) = p(x, z). It follows that the corresponding optimal generator and encoder in Eq. (3) theoretically guarantees a perfect reconstruction for x ∼ q(x) .
Recently, efforts have been made to utilize GANs for anomaly detection, especially bi-directional GANs mentioned above that can readily reconstruct an data example via the encoder and generator network. For example, Adversarially Learned Anomaly Detection (ALAD) in [23] trains ALICE model on normal data, where they add an extra discriminator Dzz to encourage cycle consistency in the latent space. The anomaly score A(x) depends on how well x can be reconstructed.
However, one important point has been overlooked in previous generative based anomaly detection approaches. Although these generative models have the ability to reconstruct normal samples and assign low anomaly score to normal data, there is no guarantee on the detection of anomalous samples. To be more specific, these methods rely on the property that the reconstruction G(E(x)) is close to x for normal data and far for anomalous; however, G(E(x)) does not necessarily yield poor reconstruction and high anomaly scores for anomalous samples. Especially for previous GAN-based models, it is not guaranteed that the discriminator, trained to distinguish between x ∼ q(x) and "fake" samples from the generator, can successfully discriminate between normal and abnormal data.

Regularize the Discriminator and Generator
To tackle the limitations mentioned above and enable the GAN to distinguish between normal and abnormal data, we propose a penalty distribution t(x) such that x ∼ t(x) are considered as adversarial examples for the discriminator during training.
Our method uses ALICE as starting point. Concretely, we propose to include a regularization term E x∼t(x) log(1 − Dxz(x, E(x))) in Eq. (2). The first part of our objective function is as follows: In our model, the penalty distribution t(x) is chosen to be a random distribution, e.g., Gaussian distribution. These random distributions have broad support so that by adversarial training, the generator and the discriminators are biased towards to the normal manifold as proved in this section later. As a result, if we consider G(E(x)) where x is anomalous, G(E(x)) will be close to the normal sample distribution. Consequently, G(E(x)) must be far from x, which is the desired outcome for our detection algorithm. These arguments are further supported by the theoretical analysis later in this section and the superior performances from our model.
Next, we present theoretical results showing that the optimal discriminator and generator distribution are biased more strongly towards normal data. We first derive the optimal discriminator and then the corresponding optimal generator. Consider the following joint distributions: The conditional distributions p(x|z) and e(z|x) are specified by the generator and the encoder networks respectively. Recall that marginal distributions q(x), t(x) and p(z) correspond to normal data distribution, penalty distribution and latent variable distribution. The following proposition shows the optimal discriminator Dxz: Proposition 1 For fixed generator G and encoder E, the optimal discriminator D * xz from Eq. (4) is given by: The proof is in the supplementary material (SM). This optimal discriminator considers both normal data distribution and penalty data distribution. This result shows that, unlike classic GANs trained only on normal data, the optimal discriminator in our model is assigning higher probability to more normal data and lower probability to anomalous data with smaller q(x). Experiments on synthetic datasets in section 5.1 further support this conclusion.
Next we will show the optimal generator distribution. Substitute Eq. (5) back to Eq. (4) and let s(x, z) = q(x, z) + t(x, z) + p(x, z) and C(E, G) = V (D * xz , G, E) for shorthand, it follows that: where DKL denotes KullbackLeibler divergence.
The proof follows Karush-Kuhn-Tucke (KKT) conditions and the convex property of β. The detailed explaination is given in the SM.
The optimal generator in Eq. (7) guarantees that anomalous data x with low q(x) has a poor reconstruction. Since βq(xi, zj) − t(xi, zj) = (βq(xi) − t(xi))e(zj|xi), this theorem indicates that the optimal generator maps the latent variable z to x for which the normal data probability q(x) is high and the penalty distribution t(x) is low. The penalty distribution t(x) is used to bias the generator more strongly towards the normal set, removing the "outliers" with low q(x), as shown in Thm. 1 and Prop. 1, and therefore improving accuracy overall. In the absence of any information about actual anomalies, this is an effective strategy.
As an example, assume that q(x) has support in a manifold of dimension less than d in a unit cube (assuming as is commonly the case that the input data is normalized). As a result q(x) is large inside this manifold and small outside. In contrast, t(x) is chosen to be roughly uniform inside the unit sphere (e.g., a Gaussian distribution with standard deviation 1 and mean 0). In this set up, whenever q(x) is large t(x) is small (it doesn't have to be close to zero) and vice versa. In that case our algorithm will provide the correct bias for G and will improve the predictions. A simple example in R 2 is shown in Fig. 1. The normal q(x) is a "narrow" Gaussian distribution and t(x) is roughly uniform in [−1, 1] 2 . Note the t(x) is unrelated to the actual anomalous examples. The resulting generator p(x) is computed by minimizing Eq. (6) using the convex optimization library CVXPY [1] (for simplicity of presentation, we assume the latent distribution p(z) and the conditional distribution e(z|x) to be constant, so p(x) ∼ p(x, z) and q(x) ∼ q(x, z)). Same as proved in Thm. 1, RCGAN correctly biases the generator distribution towards the normal manifold.

Generation with Cycle Consistency
To further guarantee a good reconstruction for normal data x ∼ q(x), we include a second discriminator Dxx as in [9] to enforce cycle-consistency of x and its reconstruction. The cycle-consistency optimization objective function is defined as: wherex = G(E(x)) is the reconstruction of x. As shown in [9], the optimal generator and encoder of the objective in Eq. (9) leads to E e(z|x) p(x|z) = δ(x −x), resulting in a perfect reconstruction for x ∼ q(x) theoretically. The optimal discriminator is D * xx (x,x) = δ(x −x). The complete minmax optimization objective of the our framework Regularized Cycle Consistent GAN (RCGAN) is the sum of Eqns. (4) and (9): After the model is trained on normal data from p(x) and advesarial data from t(x) following Eq. (10), at the detection phase, the anomaly score assigned to an example x is defined as: The anomaly score A(x) describes how well the example x is reconstructed, determined by the discriminator Dxx. As showed previously, our model enforces a large reconstruction error on anomalous data with low q(x) (which is a desirable feature for identification). Meanwhile, the cycle-consistent objective function in Eq. (9) encourages accurate reconstruction for normal data. This discrepancy endows our model with the ability to discriminate the abnormal from the normal much more reliably. In the next section, numerical experiments on both synthetic and real-world datasets will further demonstrate the effectiveness of our model.

Towards Better Discriminators
We first test on synthetic dataset in R 2 = {x1, x2}. We compare our model with another GAN-based method ALAD. Since both RCGAN and ALAD use discriminators for anomaly detection, we plot the discriminators output after training on normal data. Three cases of normal data distributions q(x) are tested on: loop, arc and four-dot. The normal data samples are shown in the first row in Fig. 2. The second and the third row contain output by discriminators in RCGAN and ALAD respectively. The output probability given by Dxz(x, E(x)) and Dxx(x, G(E(x))) over x1 ∈ [−3, 3], x2 ∈ [−3, 3] are presented in the first and second column in each case. Higher probability (represented by brighter color) indicates that the discriminator is more confident that x is normal. RCGAN and ALAD are trained using the same neural structures and hyper-parameters for fair comparison. We use the multivariate Gaussian N (0, I) as t(x) in RCGAN. Discriminators from RCGAN exhibit superior performance, assigning lower normal probability to anomalous data outside the normal data cluster. Notice in the "four-dot" case that normal data reside in discontinuous clusters; nevertheless, Dxz and Dxx in RCGAN accurately recognize anomalous data between normal data clusters. This shows that random noise from the penalty distribution t(x), serving as adversarial examples during the training, encourages discriminators to assign low probability to regions where normal examples are missing. This matches with our previous theoretical analysis.

Baseline Models
In our experiments with real-world dataset, we compare RCGAN with following baseline models: Anomaly Detection GAN (ADGAN) is the first GAN-based anomaly detection model [21]. After training a DCGAN [16] on normal data, test examples are mapped back to corresponding latent variables z by minimizing the weighted sum of reconstruction error and feature mapping error via gradient descent.
Adversarially Learned Anomaly Detection (ALAD) in [23] uses the GAN framework proposed in ALICE and exploit the encoder network to map data back to latent variable space. The anomaly score is the feature mapping error estimated from Dxx.
Deep Autoencoding Gaussian Mixture Model (DAGMM) in [25] learns an autoencoder for feature extraction and a Gaussian Mixture Model for density estimation. Data with small weighted sum of probabilities predicted by the learned Gaussian mixture are considered anomalous.
Deep Structured Energy-Based Model (DSEBM) trains a network that outputs the energy associated with a data example [24]. Two types of DSEBM, leveraging energy (DSEBM-e) and reconstruction error (DSEBM-r) respectively for anomaly detection, are included for comparison.
Deep Support Vector Data Description (DSVDD) trains a neural network while minimizing the volume of a hypersphere that encloses the network representations of the data [18]. The anomaly score is defined as the euclidean distance of the data the center of hypersphere.
Isolation Forests (IF) constructs trees by randomly selecting features and then arbitrarily choosing a split value on selected features [11]. The anomaly score of an example is defined as the averaged path length from the root node to the example.
One Class Support Vector Machines (OC-SVM) is a kernelbased method that finds maximum margin hyperplane that separates normal data from the origin [22]. We use an RBF kernel K(x, x ) = exp(− 1 m x − x 2 ) in the experiment, where m is the size of input features.
Deep Convolutional Autoencoder (DCAE) is a classical autoencoder with encoder and decoder based on convolutional neural network [12]. The anomaly score is l2 norm of the reconstruction error.

Tabular Dataset
We tested on four tabular datasets: KDDCUP, Arrhythmia, Thyroid and MUSK. Note we choose Thyroid and MUSK dataset for their "" "# "" "# "" "# loop arc four-dot RCGAN ALAD Figure 2. Results on three synthetic datasets: "loop," "arc" and "four-dot." The first row shows samples of normal data, and the second and third row show the output probability of discriminators in RCGAN and ALAD respectively. In each dataset, the left column visualizes the output probability of Dxz, and the right one shows the output from Dxx. These plots show the clear distinction between normal and abnormal sets with RCGAN. ALAD's prediction is much fuzzier. • KDDCUP. The original KDDCUP network intrusion dataset [5] contains 494,021 samples with 34 categorical and 7 continuous features. During pre-processing, categorical features are encoded using one-hot representation, and the final data examples have 121 dimensions. Data labelled as "non-intrusion" (consisting of 20% in the dataset) are treated as anomalies since they are in a minority group. In the test phase, the top 20% of test data with the highest anomaly scores A(x) are predicted as anomalies. • Arrhythmia. The Cardiac Arrhythmia dataset [5] has 452 instances with 274 attributes, and each instance is classified into one of 16 groups. The smallest classes, including 3,4,5,7,8,9,14, and 15, consist of 15% of the entire samples and are treated as the anomaly class. The remaining groups are considered as normal data. The top 15% of the test data with the highest anomaly scores A(x) are labeled as anomalies. • Thyroid. The Thyroid disease dataset from [5] is a three-class classification dataset with 3,772 instances and 6 continuous attributes. The "hyperfunction" class, consisting of 2.5% of the dataset set, is treated as anomaly. Therefore, the top 2.5% of the test data with the highest anomaly scores A(x) are inferred as abnormal. • Musk. The Musk Anomaly Detection dataset processed by [17] is originally a multi-class classification dataset on musk molecular with 3,062 instances and 166 attributes. The musk category 213 and 211 are regarded as anomalous, and the overall anomaly ratio is 3.2%.
We take 80% of data by random sampling for training and the remaining for test in KDDCUP and Arrhythmia. For Thyroid, 50% of data are randomly chosen for training, and the rest are for test. For Musk, we follow the original data split. Models are evaluated by precision, recall and F1 scores of the anomaly examples predicted. The results are summarized in Table 1. Performances of benchmark models are collected from [25] and [23], except that ALAD on Arrhythmia and DSVDD are run by us. Results for RCGAN are averaged over 10 runs. The neural structure for discriminators, generators and encoders used in our models are standard fully connected layers with non-linear gate. We will add more details on these neural in the final version where more space are given. We use N (0, I) as t(x) in RCGAN.
For a clearer comparison, we also provide the error bar for RC-

Image Dataset
We further test on the image dataset CIFAR-10 as a novelty detection task. Ten distinct datasets are generated by regarding each image category as the normal class. We follow the train/test split in the original dataset. The metrics for evaluation is area under the receiver operating curve (AUROC), averaged on 10 runs. We introduce another type of anomaly score function, proposed in ALAD [23]: where lxx denotes the last layer before the logit output in the discriminator Dxx. A(x) is motivated by the matching loss used to stabilize training of GANs in [20]. Again, our model shows an overall strong performance and achieves highest mean performance in five datasets with statistically significant margins. Notably, RCGAN outperforms baseline models on the average performance across 10 datasets (shown in the last row). Compared with previous GANbased anomaly detection algorithm, ADGAN and ALAD, RCGAN shows overall competitive results. The hyper-parameters and neural structures of RCGAN closely follow ALAD for a fair comparison. More details on training will be provided in the final version.

Discussion
Ablation Study. To further demonstrate the effectiveness of leveraging the penalty distribution t(x), we test RCGAN with and without t(x) in the adversarial training (by removing the last term in Eq. (4)). Results on Thyroid and Arrhythmia dataset are shown in Table 3. Utilizing the penalty distribution shows consistent improvement over the model without t(x). This improvement again confirms the effectiveness of our model.
Choices of the Penalty Distribution. In all experiments mentioned previously, the penalty distributions t(x) are chosen to be Gaussian distribution with zeros mean and identity variance. In this subsection, we evaluate the effect of using different types of random distributions for t(x), including two Gaussian distributions N (0, I) (used for experiments in this paper), N (0, 2I), and a uniform distribution U(−1, 1). Our model is robust and produces consistent performance improvements using anyone of these random distributions, as summarized in Table 4. Function of the Penalty Distribution. The penalty distribution introduced in our model is not designed to ensemble actual anomalous data. Our method still works if t(x) is not close to real anomalies. We still have good reconstruction for data with high q(x), i.e., high certainty that x is normal, and poor reconstruction for data with low q(x) (which is favorable). The conclusion from Thm. 1 is unrelated to actual anomalies. In the case that t(x) coincides with normal data q(x), we can systematically modify t(x) appropriately to avoid this issue, since we have samples (training data) from q(x).

Conclusion
In this paper, we propose a GAN-based anomaly detection approach which explicitly introduces the penalty distribution in the adversarial learning. Theoretical analysis shows that the introduction of the penalty distribution offers stronger guarantees that our model will correctly distinguish normal from abnormal data. In our numerical experiments, we show that our model consistently outperforms baseline anomaly detection models on four tabular datasets and ten image dataset. We also demonstrated that the performance of the algorithm is relatively insensitive to the choice of the penalty distribution. For future work, we would like to extend the usage of penalty distribution to other generative models besides GAN-based framework.

A.1 Proof of Proposition 1
In Eq. (4), the discriminator Dxz is trained to maximize the quantity Vano(Dxz, G, E), which can be rewritten using the three joint distributions above: Recall that the function a log(x)+b log(1−x) achieves its maximum at x = a a+b . Therefore, for fixed generator G and encoder E, the optimal discriminator Dx,z is: A.2 Proof of Theorem 1 Substitute Eq. (5) back to Eq. (4) and let s(x, z) = q(x, z) + t(x, z) + p(x, z) and C(E, G) = V (D * xz , G, E) for shorthand, we have: where s(x, z) = t(x, z) + p(x, z) + q(x, z). Consider the discrete case, and let pij, qij and tij denote px i ,z j , qx i ,z j and tx i ,z j . Given fixed encoder network, the optimization problem in Eq. (15) can be rewritten as: For all i, j, the first condition in Eq. (18) leads to: log tij + pij tij + pij + qij + λ − µij = 0 If pij = 0, then µij = 0 and it becomes: where β = 1 e λ −1 . Recall i,j pij = 1, then β = 1+ tmn qmn for pmn = 0.
If the solution β = 2, then from the definition of f we see that S β,c must be empty. So 2q − t ≥ 0 everywhere. The converse is true.
If β = 1, then q = 0 implies t = 0 (so that f (1) can be equal to 1; see Eq. (24)). For all indices, either q or t is 0 at the corresponding index. This can be written as qt = 0 everywhere, or equivalently the support of q and t do not overlap.