{'fraction_non_alphanumeric': 0.034506267868924564, 'fraction_numerical': 0.009588739828458323, 'mean_word_length': 4.831110541164401, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Y-GAN: Learning Dual Data Representations for Efficient Anomaly Detection

We propose a novel reconstruction-based model for anomaly detection, called Y-GAN. The model consists of a Y-shaped auto-encoder and represents images in two separate latent spaces. The first captures meaningful image semantics, key for representing (normal) training data, whereas the second encodes low-level residual image characteristics. To ensure the dual representations encode mutually exclusive information, a disentanglement procedure is designed around a latent (proxy) classifier. Additionally, a novel consistency loss is proposed to prevent information leakage between the latent spaces. The model is trained in a one-class learning setting using normal training data only. Due to the separation of semantically-relevant and residual information, Y-GAN is able to derive informative data representations that allow for efficient anomaly detection across a diverse set of anomaly detection tasks. The model is evaluated in comprehensive experiments with several recent anomaly detection models using four popular datasets, i.e., MNIST, FMNIST and CIFAR10, and PlantVillage.


Introduction
Anomaly detection represents a challenging problem, where the goal is to distinguish anomalous data from data considered to be normal 1 Perera et al. (2021). Most recent solutions approach this problem from a one-class classification perspective and attempt to learn detection models using only normal training data. Such an approach has led to successful deployment of anomaly detection techniques in a wide variety of application domains where the anomalous data is not readily available or is difficult to collect, including (visual) quality inspection Racki et al. (2018) Zhou et al. (2020). These models commonly learn some latent representation that can be used to reconstruct normal data samples with high fidelity. Because no anomalous data is seen during training, the basic assumption here is that such (anomalous) samples will lead to poor reconstructions. As a result, differences in reconstruction quality are commonly exploited to differentiate between normal and anomalous data. Reconstructive approaches have been shown to perform well across a broad range of anomaly detection tasks and to provide competitive results across several popular benchmarks Akcay et al. (2019); Akçay et al. (2019). However, as emphasized in Fei et al. (2021), the learning objectives typically utilized for learning reconstructive models predominantly focus on low-level pixel comparisons instead of image semantics intrinsic to the training data. This results in latent representations that encode low-level data characteristics that are likely to be shared between normal and anomalous data samples Dosovitskiy & Brox (2016) instead of more discriminating higher-level semantics. Additionally, when data with rich visual characteristics and complex appearances is used for training, the likelihood of high-fidelity reconstructions of anomalous data increases as well, rendering reconstruction-based models less effective in such cases.
This problem is further exacerbated by the high generalization capabilities of modern generative models, where highquality reconstructions of anomalous samples can already be expected under more relaxed assumptions Fei et al. (2021); Gong et al. (2019). The key challenge with these techniques is, therefore, to learn latent representations that encode important image semantics and are uninformative with respect to low-level visual characteristics commonly shared by normal and anomalous data.
Based on this insight, we propose in this paper a novel anomaly detection model, called 'Y-GAN,' that aims to address the above mentioned challenges. As illustrated in Fig. 1, Y-GAN is designed around a Y-shaped auto-encoder model that encodes input images in two distinct latent representations. The first representation captures semantically meaningful image characteristics useful for representing key properties of normal data, while the second encodes irrelevant, residual data characteristics. This dual encoding is enabled by an efficient disentanglement procedure that can be learned automatically in a one-class learning setting, i.e., without the use of anomalous data. To control the information content in the two latent representations, Y-GAN utilizes a latent classifier and trains it to discriminate between sub-classes/groups of normal data. In other words, it exploits differences within the normal data to learn meaningful data semantics that can later be used for anomaly detection. Additionally, a novel representation consistency loss is introduced for the training procedure of Y-GAN that ensures that the encoded information in the dual latent representations is mutually exclusive. Using this approach, Y-GAN is able to learn highly descriptive data representations that facilitate efficient anomaly detection across a variety of problem settings. The model is evaluated in extensive experiments on four anomaly detection benchmarks and compared with several state-of-the-art anomaly detection models presented recently in the literature. The results of the evaluations show that Y-GAN offers significant performance improvements over all considered competing models.
In summary, our key contributions in this paper are: • We propose Y-GAN, a novel anomaly detection method, that disentangles semantically-relevant image characteristics from residual information for efficient data representation and addresses some of the key challenges associated with reconstructive anomaly-detection models.
• We introduce a novel disentanglement strategy that enforces representation consistency and allows Y-GAN to exclude uninformative image information from the anomaly detection task in a one-class learning setting. We note that the same strategy is also applicable to other problem domains in need of efficient disentanglement.
• We show the benefit of the proposed dual data representation over several state-of-the-art anomaly detection mechanisms by reporting superior results on multiple benchmarks and across different anomaly detection tasks.
In this section, we present the most important background information with respect to such one-class models to provide the necessary context for our work. For a more comprehensive coverage of the area of anomaly detection and a broader discussion of existing solutions, the reader is referred to some of the excellent surveys in this field, e.g., Perera et al. (2021); Chalapathy & Chawla (2019);Chandola et al. (2009);Pang et al. (2021).

Reconstruction-based Anomaly Detection
Reconstruction-based models represent one of the most widely studied groups of anomaly detectors in the literature. Such models try to discriminate between normal and anomalous data by evaluating reconstruction errors produced by generative networks trained exclusively on normal data. Schlegl et al., Schlegl et al. (2017), for example, proposed to project probe samples into a GAN latent space learned in this manner in their AnoGAN model and generate reconstructions from the computed latent representation for scoring. While this approach relied on two separate steps (i.e., latent-space learning and reconstruction), later improvements, such as f-AnoGANSchlegl et al. (2019)  More recent work on reconstruction-based models capitalized on the importance of designing informative/discriminating latent spaces that can widen the gap between reconstruction errors observed with normal and anomalous data. Perera et al., , for example, designed a constrained latent space for their OCGAN model, such that only samples belonging to the class observed during training were reconstructed well, while anomalous samples were not. Zhao et al. Zhou et al. (2020) split images into two distinct parts (i.e., texture and structure) in their P-Net model. The two parts were then encoded separately with the goal of making the generated representations more informative for anomaly detection. The model was later extended in Zhou et al. (2021), with an additional module, that memorizes the correspondence between the structure and its texture. Integrated memory modules were also utilized by Park et al., Park et al. (2020) in their anomaly-detection approach, to lessen the representation capacity of their model for anomalous data and further improve detection results. Conceptually similar solutions to the work of Park et al., were presented in Gong et al. (2019); Yang et al. (2021).
The Y-GAN model proposed in this paper, follows the general idea of reconstruction-based methods, but unlike competing solutions encodes the input data in two distinct latent representations that allow for the separation of Figure 2: Overview of the proposed Y-GAN model and training loss terms. The model consist of a Y-shaped autoencoder with encoders, E s and E r , for dual data representation, a decoder D for image reconstruction, a latent classifier C for data disentanglement, and an adversarial discriminator Ds, which ensures that data reconstructionsx cannot be distinguished from the original input samples x. Using an efficient disentanglement procedure, Y-GAN aims to learn a semantically meaningful data representation in z s , that encodes only characteristics relevant for representing normal data, while capturing irrelevant, residual data characteristics in z r . The figure is best viewed in color. relevant information from information irrelevant for the anomaly detection task. As we show in the following sections, this separation of information is: (i) achieved without any assumptions regarding the source of relevant information (e.g., texture, color, structure, etc.), (ii) is learned automatically in an end-to-end manner from normal data only, and (iii) leads to significant performance improvements over existing reconstruction-based models on various anomaly detection tasks.

Anomaly Detection with Proxy Tasks
To address some of the limitations of reconstruction-based anomaly detectors, another major group of existing models uses proxy tasks when learning to discriminate between normal and anomalous data. The main idea behind solutions from this group is that models trained on normal data will fare badly in the considered proxy task when subjected to anomalous samples. Ye et al., Fei et al. (2021), for instance, explored image restoration in this context and showed that differences in restoration performance can be used for anomaly detection. Noroozi et al. Noroozi & Favaro (2016)  El-Yaniv (2018) investigated classification objectives defined over self-annotated recognition problems to facilitate anomaly detection. The proposed Y-GAN is related to these models in that it also relies on a proxy classifier, which, however, aims at distinguishing between different sub-groups of the normal data. By defining the proxy task as a classification problem over normal data, Y-GAN is able to: (i) ensure a compact representation of the normal data in the model's latent space, and (ii) automatically learn semantically relevant information for the anomaly detection task.
Both of these characteristics are beneficial for anomaly detection performance, as we demonstrate in the experimental section.

Pre-trained Models for Anomaly Detection
In an effort to capture the most discriminating characteristics of the input data, many recent anomaly detection techniques try to leverage the representational power of features extracted by pre-trained (large-scale) classification models. Defard et al., Defard et al. (2021) and Rippel et al., Rippel et al. (2020), for example, have shown that such features can be successfully used for explicitly modelling the distribution of normal samples. Anomalies are in this case detected as out-of-distribution samples. Similar solutions based on shallower models have also been investigated in literature, e.g., Cohen & Hoshen (2020); Roth et al. (2021). Reiss et al., Reiss et al. (2021) proposed to finetune existing pretrained models using normal training data and showed that such an approach leads to impressive anomaly detection performance. Mai et al., Mai et al. (2021) explored the use of knowledge distillation to limit the set of pretrained features considered when building anomaly detection models. Similar ideas have also been pursued in Bergmann et al. (2020); Wang et al. (2021a); Salehi et al. (2021). Wang et al., Wang et al. (2021b) extended these studies by fine-tuning the distilled student network and further improved the detection rates on several anomaly detection benchmarks. In contrast to the outlined solutions, Y-GAN does not rely on pre-trained models for data representation, but instead learns discriminating encodings from scratch by separating uninformative data characteristics from meaningful data semantics relevant with respect to the normal training data. By doing so, it is able to selectively encode part of the characteristics that are relevant for discrimination without the need for large-scale datasets and resource hungry (pre-trained) classification models.

Methodology
The main contribution of this work is a novel (powerful) anomaly detection model, called Y-GAN. In this section we present the proposed model in detail and describe its main characteristics.

Proposed Model
Y-GAN, illustrated in Fig. 2, represents a generative adversarial network built around a Y-shaped auto-encoder.
The key idea behind Y-GAN is to split the latent space of the auto-encoder into two distinct parts by disentangling informative image semantics (e.g., shapes, appearances, textures), relevant with respect to some normal training data from uninformative, residual image information (e.g., noise, background, illumination changes). This separation of image content is achieved through an efficient disentanglement procedure (facilitated by a latent classifier) and allows our model to learn highly descriptive data representations for anomaly detection even in the challenging one-class learning regime. Details on the individual components of Y-GAN are given below.
The Auto-Encoder Network. To be able to separate relevant image content from residual information, we design a Y-shaped auto-encoder network and use it as the generator for Y-GAN. As illustrated in Fig. 2, this Y-shaped network consist of two separate encoders and a single decoder. The two encoders are identical from an architectural point of view, but have distinct parameters that can be learned independently one from the other. The first encoder E s maps the input image x ∈ R w×h into a semantically-relevant latent representation z s and the second E r maps x into the dual, residual representation z r , i.e.: where d stands for the dimensionality of z s and z r . The complete latent representation of x is computed as a concatenation of the two partial representations, i.e., z = z s ⊕ z r , and passed to the decoder D for reconstruction, i.e., To ensure that all of the image content in x is captured by the concatenated representation z, we use a standard L 1 reconstruction loss L rec when learning the parameters of the auto-encoder Isola et al. (2017): Moreover, an adversarial loss term and additional learning objectives that control the information content in the latent representations z s and z r are also utilized during training. We discuss these in the following sections.
The Discriminator. The expressive power of the latent representations, z s and z r , critically depends on the fidelity of the image reconstructionsx. To improve fidelity and ensure that the reconstructed samples follow the distribution of the normal training data, we include a discriminator Ds in the training procedure of Y-GAN and use an additional adversarial loss L adv when learning the model. Following the recommendations from Salimans et al.
(2016), we update the weights of the auto-encoder, i.e., the generator of Y-GAN, based on the following featurematching objective that reduces training instability and avoids GAN over-training, i.e.: where f (·), in our case, denotes the activations of the last convolutional layer of Ds. Conversely, we encourage the discriminator to distinguish between real and fake images by optimizing a standard binary cross-entropy loss: The Latent Classifier. The Y-shaped design of Y-GAN's auto-encoder allows for the partitioning of the latent space into two representations, z s and z r . We force these representations to encode mutually exclusive information by using a disentanglement procedure based on a latent classifier C. The goal of this classifier is two-fold: (i) to encourage semantic information relevant for representing normal data to be encoded in z s and (ii) to force the irrelevant residual information into the latent representation z r .
To be able to learn C, we assume that the normal training data can be partitioned into N sub-classes 2 . The classifier is then trained to predict correct class labels from the latent representations z s and to misclassify input 2 Note that this is a reasonable assumption for many applications and holds for a wide variety of datasets and experimental protocols from the  in two latent spaces that need to encode mutually exclusive information. Y-GAN ensures that the representation in the latent representation z s is independent of that in z r by encouraging the model to produce the same z s even when changes in z r are introduced. Shown is an illustrative toy example involving digit shapes (assumed to be relevant) and background textures (assumed to be irrelevant). samples x given their latent representation z r . This training procedure controls the information content in the dual latent representations and helps to learn meaningful data characteristics for anomaly detection without examples of anomalous data.
A cross-entropy loss is utilized to maximize the classification performance of C based on the semantically-relevant latent representation z s , i.e.: where y ∈ R N is the one-hot encoded ground truth label of x andŷ s = C(E s (x)) ∈ R N is the classifier prediction.
While minimizing L s forces the latent representation z s to be informative with respect to the classification of the normal data, our goal is to achieve the opposite for z r . To this end, we transform z r with a gradient reversal layer Ganin & Lempitsky (2015). This transformation layer R acts as an identity function in the forward pass through the model, i.e., R(z r ) = z r , but reverses the gradients from the subsequent layer during back-propagation, i.e., where λ R is a hyper-parameter and I ∈ R d×d is an identity matrix. The minimization of relevant information content in z r is then achieved through the following objective: Enforcing Representation Consistency. The loss functions in Eqs. (6) and (8) provide for a first level of disentanglement, but do not completely prevent information leakage between the latent representations z s and z r . For this purpose, we introduce a novel consistency loss (L con ) that penalizes the encoder E s in case it extracts inconsistent (semantically-relevant) information in z s , when changes in the residual representation z r are introduced. The overall idea of this procedure is illustrated on the right part of Fig. 2 and using a simple visual example in Fig. 3.
To calculate L con , we first randomly shuffle the set of residual representation z r , generated from the samples in a given training batch, so that each z s vector is concatenated with a z r vector, belonging to a randomly chosen sample from the batch. These artificially created concatenations are then passed to the decoder D, which generates hybrid reconstructionsx , such thatx = D(z s ⊕ z r ). Next, the reconstructed images are fed to the encoder E s , which is expected to extract latent representationsẑ s that are equivalent to the vector z s , initially used for the generation of the hybrid reconstructionsx . An angular dissimilarity measure is used to penalize differences between z s andẑ s , i.e.: By minimizing L con , we encourage the encoder E s to extract image information that is invariant to the residual data characteristics encoded in z r . Note that, L con is calculated and optimized for each batch in the training set separately.
The size of the training batches, therefore, has to be at least equal or greater than two, otherwise L con has no effect. It also worth noting that L con does not control what is encoded in the latent representations z s and z r but only ensures that the information in the two representations is mutually exclusive, or in other words, that the representations are properly disentangled.

Y-GAN Training
Y-GAN is trained in an end-to-end fashion, using a multi-step procedure. For each training batch, the losses from Eqs.
(3) to (6) are calculated first. Next, the set of residual representations z r in the given batch is randomly shuffled and processed, as described above for the calculation of the consistency loss L con in Eq. (9). Finally, the weights of the generator (i.e., the Y-shaped auto-encoder) are updated, based on the combined objective L G , i.e.: Similarly, the weights of the adversarial discriminator Ds are updated based on the combined loss L D , i.e.: The generator and discriminator are updated alternately for a fixed number of epochs during training.

Anomaly Detection with Y-GAN
Similarly to Golan & El-Yaniv (2018), predictions of the latent classifier C are used to calculate anomaly scores.
Given a probe sample x, the latent representation z s = E s (x) is first computed and passed to the classifier C. Next, the activations of the output layer of the classifier are normalized Finally, the highest of these "probabilities" is used to compute the anomaly score s, i.e.: for the given test sample x. Due to the normalization procedure, the generated anomaly scores are bounded to s ∈ [0, 1], with 0 representing ideal normal data. Note that for the calculation of the anomaly scores only E s and C are needed, which significantly shortens inference time.

Datasets
We evaluate Y-GAN on three standard anomaly detection benchmarks, i.e., MNIST LeCun et al. (1998); LeCun et al., FMNIST Xiao et al. (2017), and CIFAR10 Krizhevsky (2009) • CIFAR10 contains 60, 000 color images of size 32 × 32 pixels representing animals and vehicles from 10 categories, i.e., airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, trucks. Different from MNIST or FMNIST, images in CIFAR10 do not have a uniform background and exhibit considerable diversity in terms of appearance even within the same category. These characteristics make it particularly challenging for anomaly detection tasks Krizhevsky (2009).
• PlantVillage is a recent real-world dataset of leafs and contains 54, 305 color images of 256 × 256 pixels. Each image represents a single leaf, photographed on a homogeneous background. Images are divided into 14 unbalanced categories of different plant species, 9 of which contain both, healthy and ill leafs. 3 categories contain only healthy samples, while the remaining 2 have no disease-free leafs. Plant diseases are usually manifested as changes in the shape and the color of the leaf, but can also appear as a subtle pattern, that covers the leaf base, as shown in Fig. 5.
The selected datasets allow for the evaluation of anomaly detection models in different problem settings, i.e.: In PlantVillage, we also report true positive (TPR) and true negative rates (TNR) for each class in the dataset. Both metrics are calculated at the equal error rate (EER) point of the ROC curve generated for all training samples from the dataset.

Implementation Details
Model Architecture. Y-GAN consists of five architectural building blocks, i.e., two encoders, E s and E r , a decoder, D, a discriminator, Ds and a classifier, C. The first four components are designed after DCGAN Radford et al. (2016), whereas the topology of the classifier is determined experimentally.
The two encoders, E s and E r , consist of convolutional layers with stride 2. Each convolutional layer is followed Training Setting. The learning objectives in Eqs. (10) and (11)  The weights in L G and L D are determined empirically through an optimization procedure on validation data. For the experiments we use λ 1 = λ 5 = 50, and λ 2 = λ 3 = λ 4 = λ 6 = 1. While these weights are kept constant, the weight associated with the gradient reversal layer is initialized to a value of λ R = 0 and is then gradually increased as the training progresses, as suggested in Ganin & Lempitsky (2015). All models are trained for 100 epochs on MNIST, FMNIST, and CIFAR10 and for 200 epochs on PlantVillage, where less data data is available for the learning procedure.
Implementation. Y-GAN is implemented in Python 3.7. using PyTorch 1.5. and CUDA 10.2. All source 3 Recall that 2 out of the total of 14 classes have only anomalous samples and are not considered during training.
code, model definitions, and trained weights have been made publicly available to facilitate reproducibility 4 . Using a personal desktop computer with an Intel ® Core TM i7-8700K CPU and an NVIDIA ® GeForce RTX 2080 Ti GPU, it takes around four hours to train Y-GAN on MNIST, FMNIST, and CIFAR10. For the higher resolution images in PlantVillage, the training stage takes around five hours. Once the model is learned, a single image is processed in around 2.6 ms for the smallest 32 × 32 images and 13.5 ms for the largest 256 × 256 images.

Proof-of-Concept Study
To explore the characteristics of the dual latent-space representation and evaluate the effectiveness of the disentanglement process performed by Y-GAN, we first conduct a proof-of-concept study. To this end, we generate a color version of MNIST (Color-MNIST hereafter) by inverting the thresholded black and white images of the dataset and replacing the white background with one of the following colors: red, green, blue, cyan, yellow, purple, violet, brown, dark green, or orange. We make sure all colors are represented equally in the generated dataset. Next, we train Y-GAN on the constructed dataset by considering digits 1 to 9 as normal data, and 0 as anomalous. In the test phase, we present the model with unseen samples and compute their latent representations, z s and z r . Finally, we generate hybrid reconstructions by concatenating the latent vector z s taken from one test sample with the latent vector z r of another randomly selected test sample and pass the concatenated vector through the decoder. Example reconstructions produced with this process are shown in Fig. 6(a).
As can be seen, Y-GAN learns to disentangle data characteristics relevant for the digit representation task, from characteristics that are irrelevant, i.e., background color in this toy example. Consequently, the replacement of the original latent vector z r causes a change in the background color, which is now inherited, from the randomly selected sample -shown on the left part of Fig. 6(a). Meanwhile, the shape of the digit in the original image is preserved well. (2008) to visualize the distribution of the generated data in the dual latent spaces in Fig. 6(b). Here, 250 random samples of each of the Color-MNIST digit classes are used for visualization. Note that for the semantically-relevant latent space, samples corresponding to digits 1 to 9 form compact and well separated clusters (marked z s ), while samples for the anomalous 0 are considerably less compact despite the fact that they come from a single (homogeneous) class. Nevertheless, they do not overlap (significantly) with the normal data. In the residual latent space (marked z r ), 10 clusters corresponding to the background colors used in Color-MNIST can be identified in the t-SNE plot. However, each cluster contains samples from all 10 digits, suggesting that this representation has limited discriminating power for anomaly detection.

Next, we use t-distributed Stochastic Neighbor Embeddings (t-SNE) Van der Maaten & Hinton
To test the behavior of Y-GAN on more complex data, we repeat the same experiment using CIFAR10. Selected synthesized samples and their respective source images are shown in Fig. 7. We again observe that Y-GAN learned to successfully disentangle semantically relevant attributes from those that are irrelevant for representing classes in the normal data. The shape of the objects and other visually important characteristics are obviously inferred from  the source image of the relevant latent vector z s , while background style and colors and inferred from the residual latent vector z r . Different from the Color-MNIST example, where the digits represent homogeneous classes with limited variability, the large intra-class variability of CIFAR10 images leads to lower quality reconstructions, which is expected given Y-GAN's learning objectives. Nevertheless, the results validate that meaningful separation of information content is achieved in the latent space even with challenging input images.

Results and Discussion
To illustrate the performance of Y-GAN, we report in this section results that: (i) compare Y-GAN to state-of-theart techniques from the literature, (ii) were generated through a comprehensive ablation study and demonstrate the contribution of various components of Y-GAN, (iii) highlight some of the model's characteristics, and (vi) investigate the behavior of Y-GAN in a qualitative manner.

Quantitative Evaluation
We evaluate Y-GAN in comparative experiments with several state-of-the-art anomaly-detection models.    MNIST. The MNIST results, reported in  CIFAR10. Images in CIFAR10 were captured in unconstrained settings, which makes this dataset extremely challenging, as evidenced by the results in Table 3. All competing models result in mean AUC scores close (or below) to 0.6, which speaks of the difficulty of learning meaningful representations on CIFAR10. The dual representation strategy of Y-GAN, on the other hand, yields a mean AUC score of 0.763, improving on the runner-up, ARNet, by more than 25%. The proposed model also convincingly outperforms all competing models in all 10 experimental runs.
PlantVillage. Results for the PlantVillage dataset are reported in Table 4. As can be seen, Y-GAN is again the top performer with an AUC of 0.962, outperforming the state-of-the-art runner-up GANomaly by more than 23%.
Y-GAN exceeds the detection accuracy of other models with respect to normal and anomalous samples and generates fewer misses on average as shown by the TPR and TNR scores. However, we also observe that the (global) calibration of the models results in unbalanced TPR and TNR values for certain classes (e.g., Corn, Grape, Potato). Despite these calibration issues, Y-GAN performs best on average even if the individual TPR and TNR scores are considered. We attribute this performance to the implemented dual data representation strategy, that allows for excluding irrelevant data characteristics when deciding whether a sample is anomalous or not.

Ablation Study
To demonstrate the importance of different components of Y-GAN, we perform a two-part ablation study, where we first remove various parts of the learning objective L G , and then ablate parts of the model architecture.
Impact of Loss Terms. For the first part of the ablation study, three Y-GAN variants are trained with different versions of the generator loss L G : (i) L G without the consistency loss L con (A1), (ii) L G without the residual information loss L r (A2), and (iii) L G without both L con and L r (A3). The results in Table 5 show that the removal of both loss terms causes a considerable drop in the AUC scores on FMNIST, CIFAR10, and PlantVillage. Compared to MNIST, where a smaller drop is observed, these three datasets contain a greater amount of residual, semanticallyirrelevant information (e.g., various clothing prints, background style, etc.). In such cases, both disentanglement terms, L con and L r , play a significant role in the extraction of semantically-relevant information. Although the two losses are complementary, L r results in a slightly greater performance drop than L con when removed from the training objective. These results suggest that all loss terms are important and contribute to the performance of Y-GAN.
Impact of Architecture. Y-GAN uses a dual encoder to generate latent representations. Four versions of the model are implemented to highlight the importance of the design choices made around this topology: (i) a model without the dual encoder (a single encoder E is used), where the generated latent representation z = E(x) ∈ R 2d is split into two equally-sized vectors, z s ∈ R d and z r ∈ R d , on top of which L G (10) is applied (B1), (ii) the model from B1 but with a single (entangled) latent representation -no z r and associated losses (L r , L con ) are used (B2), (iii) the model from B2 but without the classifier C, i.e., an auto-encoder with a reconstruction-based anomaly score (B3), and (iv) the proposed Y-GAN model without the adversarial discriminator Ds, i.e., a dual encoder generator trained without L adv (B4). The results in Table 5 suggest that the removal of the dual encoder increasingly impacts results, as the complexity of the data (from an anomaly detection point of view) increases. The Y-shaped architecture, thus, contributes to a more efficient disentanglement of semantically-relevant and residual information. It can also be seen (from B2) that using a single entangled latent space representation is detrimental for the performance of the anomaly detection task, especially for the more challenging CIFAR10 and PlantVillage datasets. The disentanglement of irrelevant information and its removal from the decision-making process is, hence, critical for the success of Y-GAN. The exclusion of the classifier also causes a large decrease in the overall anomaly detection accuracy across all datasets, suggesting that steered representation learning is key for Y-GAN -see B3 results. Finally, training the originally proposed Y-GAN auto-encoder without the adversarial discriminator, seems to have the least significant impact on the overall detection accuracy, in comparison to other Y-GAN components (B4). Nevertheless, it does contribute to the quality of the generated image reconstructions, which can further affect the performance of the disentanglement process in more complex images, e.g., in CIFAR10 and PlantVillage.

Model Characteristics
Unlabeled Normal Data. Y-GAN assumes that the (normal) training data comes from multiple sub-classes/groups and that labels for these sub-classes are readily available. This assumption allows for the inclusion of the latent classifier C in the training process, which was shown to be critical for the overall performance of Y-GAN. Here, we show that it is possible to relax this assumption and train Y-GAN with unlabeled training data in a completely unsupervised manner. To this end, we run a clustering procedure over the training data and generate weak labels Color coding: A -learning objective ablation, B -architecture ablation  & Pentland (1991) and cluster the data with k-means. We determine the optimal number of clusters based on the average silhouette method Kaufman & Rousseeuw (2009). Finally, we utilize the generated cluster assignments as weak labels for Y-GAN training.
• K-Classes-Out Results. On MNIST, FMNIST, and CIFAR10, the clustering procedure identifies either 9 or 10 clusters in any given experimental run (9 classes are in fact represented). An analysis of the generated clusters shows that 95% of MNIST samples in each cluster share the same ground truth. This percentage is bit lower in FMNIST and CIFAR10, where it equals 91% and 87%, respectively. This suggests that the clustering reasonably well approximates the actual data classes, but also that part of the data is not assigned correct class labels. A comparison between Y-GAN trained with the ground truth labels and the weak labels generated with the clustering procedure (denoted as Y-GAN*) is presented in Table 6. As can be seen, the weak labels result in slight performance  Table 6: Mean AUC scores generated with Y-GAN models generated with: (i) ground truth labels on the sub-class structure of the normal training data, and (ii) weak labels generated through a k-means clustering procedure. Note that even in a completely unsupervised setting, where (noisy) weak labels are inferred directly from the normal training data, Y-GAN generates state-of-the-art results that outperform all of the competing models on all four experimental datasets.
different sub-classes of normal training data is beneficial for performance, even if the sub-classes are not necessarily homogeneous and contain label noise.
• PlantVillage Results. For this dataset, 12 distinct clusters are identified by k-means, which is corresponds to the number of original categories that include non-anomalous/normal samples. However, the partitioning of the PlantVillage data is in this case less accurate in comparison to the k-classes-out datasets, due to similarities between objects from different classes. After the clustering process, only 80% of PlantVillage images representing the same plant species and share the same ground truth label. Although such weak labels decrease the anomaly detection performance by approximately 12%, Y-GAN* learned without any supervision still outperforms the second best competing model, GANomaly, by 7.7% (see Table 4 for reference). Overall, these results support the observation that a competitive Y-GAN model can be trained even without access to ground truth class labels for the normal training data.
Anomaly Score Analysis. The proposed Y-GAN uses an anomaly score derived from the output of the latent classifier C to detect anomalous data. However, previous work has used other definitions of anomaly scores, including distances between the input images and their reconstructions or L p norms over latent representations among others Perera et al. (2019); Akçay et al. (2019). In this section, we compare the score utilized with Y-GAN to several other possibilities. These experiment are meant to provide additional insight into the model and the characteristics of various representations generated. For the analysis, we only use the CIFAR10 and PlantVillage datasets, which contain more complex data than the other two datasets. We implement the following competing anomaly scores for the comparison: • Image score, s x : where the anomaly score is computed-based on the reconstruction quality. Here, x is the input image andx is the Note that the proposed anomaly score achieves the strongest performance on both datasets.
reconstruction generated by Y-GAN.
• Latent score, s z : where the anomaly score is computed in the latent space using the combined latent representations z = E s (x) ⊕ E r (x) andẑ = E s (x) ⊕ E r (x). ⊕ is a concatenation operator.
• Semantic latent score, s zs : where the score is computer-based on the semantic latent space representation only, i.e., z s = E s (x) andẑ s = E s (x).
• Prototype-based semantic latent score with ground truth labels, s zp : where the (semantically-meaningful) latent probe vector z s is compared to the class prototypes z (Ci) s = 1/|C i | zs∈Ci z s , computed for the N sub-classes of the normal training data {C i } N i=1 and the minimum distance is used as the anomaly score. | · | is the cardinality of the class.
• Prototype-based semantic latent score with weak class labels, s zw : where the latent probe vector z s is compared to the class prototypes z (C * i ) s = 1/|C * i | zs∈C * i z s , computed for the N sub-classes of the normal training data {C * i } N i=1 defined through k-means clustering. The minimum distance over all class protoypes is used as the anomaly score.
• Classifier uncertainty, s c : where p = [p 1 , p 2 , . . . , p N ] ∈ R N is the probability distribution for the N sub-classes of the normal data computed by subjecting the output of the latent classifier C to a softmax function given an input probe sample x.
We note that all latent representations used in the above definitions of latent scores are normalized to unit norm prior to score calculation. The results, presented in Fig. 8, show that a simple reconstruction-based score (s x ) results in modest performance in both datasets. The latent space score s z is slightly more informative on CIFAR10, but generates weaker results on image from PlantVillage. If the residual latent space is removed from the decision making process, we observe additional improvements on both datasets. Thus, the anomaly score defined in the semantically meaningful latent space s zs already ensures better results on CIFAR10 than all of the competing state-of-the-art models evaluated in Table 3 and yields comparable detection results as a large portion of the tested models on PlantVillage. If anomaly scores are defined by also considering the sub-classes present in the (normal) training data (i.e., s zp , s zw , s c , and the proposed Y-GAN score s), we see another significant jump in AUC results on both datasets, which suggest that the structure (or distribution) of the normal data is an important source of information that can be exploited to improve anomaly detection performance.

Visual/Qualitative Evaluation
Grad-CAM Analysis. We conduct a qualitative analysis to gain better insight into the behavior of the proposed Y-GAN model. To this end, we generate Grad-CAM visualizations Selvaraju et al. (2017) of image regions that are most informative with respect to the anomaly detection task. This can be done due to the use of the latent (proxy) classifier C utilized for the computation of anomaly scores. Examples of correctly classified normal and anomalous samples (marked red) with superimposed heatmaps are shown in Fig. 9. As can be seen, the global appearance of the objects is critical for anomaly detection on datasets with homogeneous backgrounds and small intra-class variability, such as MNIST and FMNIST. Conversely, detection on datasets with more complex visual data (such as CIFAR10) is primarily based on local object and texture characteristics. Different from MNIST, FMNIST, and CIFAR10, PlantVillage exhibits relatively large intra-class variability in terms of the size, shape, illumination, and orientation of the leaves in the images. Additionally, anomalies can appear either as inconsistencies in the overall shape and color or, alternatively, impact the texture of the leafs at an arbitrary spatial location in this dataset. Therefore, both global and local image characteristics appear to play an important role in the detection of anomalous leaves, Figure 11: Sample images of undetected normal and anomalous samples (marked red) from the PlantVillage dataset. Errors with the normal data occur due to uncommon leaf shapes, holes in healthy leafs, and unusual positions/orientations. With anomalous samples, problems appear due to subtle, often unnoticeable changes in leaf color or local textures.
as seen in Fig. 9. Interestingly, Y-GAN is able to adapt to the anomaly detection task and learn descriptive and informative features from the input data regardless of whether these features correspond to global or local (or both) image characteristics.
Visual Evaluation. To better understand why the model fails to classify certain normal and anomalous samples, we perform an additional visual inspection of a few edge cases from the four experimental datasets. in Figs. 10 we show results for the k-classes out datasets MNIST, FMNIST, and CIFAR10, where the class listed below the images was considered anomalous with the presented examples. As can be seen, the undetected normal samples correspond to objects with uncommon appearance for the considered normal data, i.e., oddly-shaped digits for MNIST, ambiguous fashion classes for FMNIST, and unusual object appearancs for CIFAR10. Difficult anomalous samples, conversely, often resemble certain classes from the normal data or exhibit ambiguous appearance. Fig. 11 present edge cases for the PlanVillage dataset. Here, severely folded healthy leaves and distorted leaf shapes are often detected as anomalous. Similar outcomes are also observed with leaves with holes, although such holes do not necessarily indicate an illness. Shadows darkening various parts of non-anomalous leaves can also trick the model into misclassifying normal samples. Conversely, undetected anomalies typically represent subtle, unnoticeable changes in the leaf color or local textures.

Conclusion
The paper introduced a reconstruction-oriented auto-encoder based anomaly detection model, called Y-GAN.
Different from competing approaches, Y-GAN learns to disentangle image characteristics that are relevant for representing normal data from irrelevant residual data characteristics and derives anomaly scores from selectively encoded image information. The model was shown to significantly outperform several state-of-the-art anomaly detection models on the MNIST, FMNIST, CIFAR10 and PlantVillage datasets and provide the most consistent performance across different anomaly detection tasks among all tested models. As part of our future work, we plan to extend the model, so it allows for additional functionality, such as anomaly localization/segmentation, which is of interest for various anomaly detection tasks.