{'fraction_non_alphanumeric': 0.039499105545617176, 'fraction_numerical': 0.013953488372093023, 'mean_word_length': 4.930617441120305, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Open Compound Domain Adaptation

A typical domain adaptation approach is to adapt models trained on the annotated data in a source domain (e.g., sunny weather) for achieving high performance on the test data in a target domain (e.g., rainy weather). Whether the target contains a single homogeneous domain or multiple heterogeneous domains, existing works always assume that there exist clear distinctions between the domains, which is often not true in practice (e.g., changes in weather). We study an open compound domain adaptation (OCDA) problem, in which the target is a compound of multiple homogeneous domains without domain labels, reflecting realistic data collection from mixed and novel situations. We propose a new approach based on two technical insights into OCDA: 1) a curriculum domain adaptation strategy to bootstrap generalization across domains in a data-driven self-organizing fashion and 2) a memory module to increase the model's agility towards novel domains. Our experiments on digit classification, facial expression recognition, semantic segmentation, and reinforcement learning demonstrate the effectiveness of our approach.


Introduction
Supervised learning can achieve competitive performance for a visual task when the test data is drawn from the same underlying distribution as the training data. This assumption, unfortunately, often does not hold in reality, e.g., the test data may contain the same class of objects as the training data but different backgrounds, poses, and appearances [41,46].
The goal of domain adaptation is to adapt the model learned on the training data to the test data of a different distribution [41,34,12]. Such a distributional gap is often formulated as a shift between discrete concepts of well defined data domains, e.g., images collected in sunny weather versus those in rainy weather. Though domain generalization [24,22] and latent domain adaptation [16, * Equal contribution. 11] have attempted to tackle complex target domains, most existing works usually assume that there is a known clear distinction between domains [12,8,48,30,42].
Such a known and clear distinction between domains is hard to define in practice, e.g., test images could be collected in mixed, continually varying, and sometimes never seen weather conditions. With numerous factors jointly contributing to data variance, it becomes implausible to separate data into discrete domains.
We propose to study open compound domain adaptation (OCDA), a continuous and more realistic setting for domain  Figure 1 and Table 1). The task is to learn a model from labeled source domain data and adapt it to unlabeled compound target domain data which could differ from the source domain on various factors. Our target domain can be regarded as a combination of multiple traditionally homogeneous domains where each is distinctive on one or two major factors, and yet none of the domain labels are given. For example, the five well-known datasets on digits recognition (SVHN [33], MNIST [21], MNIST-M [7], USPS [19], and SynNum [7]) mainly differ from each other by the backgrounds and text fonts. It is not necessarily the best practice, and not feasible under some scenarios, to consider them as distinct domains. Instead, our compound target domain pools them together. Furthermore, at the inference stage, OCDA tests the model not only in the compound target domain but also in open domains that have previously unseen during training. In our OCDA setting, the target domain no longer has a predominantly uni-modal distribution, posing challenges to existing domain adaptation methods. We propose a novel approach based on two technical insights into OCDA: 1) a curriculum domain adaptation strategy to bootstrap generalization across domain distinction in a data-driven self-organizing fashion and 2) a memory module to increase the model's agility towards novel domains.
Unlike existing curriculum adaptation methods [56,6,29,25,58,57] that rely on some holistic measure of instance difficulty, we schedule the learning of unlabeled instances in the compound target domain according to their individual gaps to the labeled source domain, so that we solve an incrementally harder domain adaptation problem till we cover the entire target domain.
Specifically, we first train a neural network to 1) discriminate between classes in the labeled source domain and to 2) capture domain invariance from the easy target instances which differ the least from labeled source domain data. Once the network can no longer differentiate between the source domain and the easy target domain data, we feed the network harder target instances, which are further away from the source domain. The network learns to remain discriminative to the classification task and yet grow more robust to the entire compound target domain.
Technically, we must address the challenge of characterizing each instance's gap to the source domain. We first extract domain-specific feature representations from the data and then rank the target instances according to their distances to the source domain in that feature space, assuming that such features do not contribute to and even distract the network from learning discriminative features for classification. We use a class-confusion loss to distill the domain-specific factors and formulate it as a conventional cross-entropy loss with a randomized class label twist.
Our second technical insight is to prepare our model for open domains during inference with a memory module that effectively augments the representations of an input for classification. Intuitively, if the input is close enough to the source domain, the feature extracted from itself can most likely already result in accurate classification. Otherwise, the input-activated memory features can step in and play a more important role. Consequently, this memory-augmented network is more agile at handling open domains than its vanilla counterpart.
To summarize, we make the following contributions. 1) We extend the traditional discrete domain adaptation to OCDA, a more realistic continuous domain adaptation setting. 2) We develop an OCDA solution with two key technical insights: instance-specific curriculum domain adaptation for handling the target of mixed domains and memory augmented features for handling open domains. 3) We design several benchmarks on classification, recognition, segmentation, and reinforcement learning, and conduct comprehensive experiments to evaluate our approach under the OCDA setting.

Related Works
We review literature according to Table 1. Unsupervised Domain Adaptation. The goal is to retain recognition accuracies in new domains without ground truth annotations [41,46,49,38]. Representative techniques include latent distribution alignment [12], backpropagation [7], gradient reversal [8], adversarial discrimination [48], joint maximum mean discrepancy [30], cycle consistency [17] and maximum classifier discrepancy [42]. While their results are promising, this traditional domain adaptation setting focuses on "one source domain, one target domain", and cannot deal with more complicated scenarios where multiple target domains are present. Figure 2: Overview of disentangling domain characteristics and curriculum domain adaptation. We separate characteristics specific to domains from those discriminative between classes. It is achieved by a class-confusion algorithm in an unsupervised manner. The teased out domain feature is used to construct a curriculum for domain-robust learning.
Latent & Multi-Target Domain Adaptation. The goal is to extend unsupervised domain adaptation to latent [16,51,32] or multiple [11,9,54] or continuous [2,13,31,50] target domains, when only the source domain has class labels. These methods usually assume clear domain distinction or require domain labels (e.g. test instance i belongs to the target domain j), but this assumption rarely holds in the real-world scenario. Here we take one step further towards compound domain adaptation, where both category labels and domain labels in the test set are unavailable.
Open/Partial Set Domain Adaptation. Another route of research aims to tackle the category sharing/unsharing issues between source and target domain, namely open set [37,43] and partial set [55,3] domain adaptation. They assume that the target domain contains either 1) new categories that don't appear in source domain; or 2) only a subset of categories that appear in source domain. Both settings concern the "openness" of categories. Instead, here we investigate the "openness" of domains, i.e. there are novel domains existing that are absent in the training phase.
Domain Generalized/Agnostic Learning. Domain generalization [52,23,22] and domain agnostic learning [39,5] aim to learn universal representations that can be applied in a domain-invariant manner. Since these methods focus on learning semantic representations that are invariant to the domain shift, they largely neglect the latent structures inside the target domains. In this work, we explicitly model the latent structures inside the compound target domain by leveraging the learned domain-focused factors for curriculum scheduling and dynamic adaptation.

Disentangling Domain Characteristics
We separate characteristics specific to domains from those discriminative between classes. They allow us to construct a curriculum for increment domain adaptation.
We first train a neural network classifier using the labeled source domain data {x i , y i } i . Let E class (·) denote the encoder up to the second-to-the-last layer and Φ(E class (·)) the classifier. The encoder captures primarily the classdiscriminative representation of the data.
We assume that all the factors not covered by this class-discriminative encoder reflect domain characteristics. They can be extracted by another encoder E domain (·) that satisfies two properties: 1) Completeness: Decoder(E class (x), E domain (x)) ≈ x, i.e., the outputs of the two encoders shall provide sufficient information for a decoder to reconstruct the input, and 2) Orthogonality: the domain encoder E domain (x) shall have little mutual information with the class encoder E class (x). We leave the algorithmic details for meeting the first property to the appendices as they are not our novelty.
For the orthogonality between E domain (x) and E class (x), we propose a class-confusion algorithm, which + .

Source Domain Images
Class Memory Domain-Focused Factors Figure 3: Overview of the memory-enhanced deep neural network. We enhance our network with a memory module that facilitates knowledge transfer from the source domain to target domain instances, so that the network can dynamically balance the input information and the memory-transferred knowledge for more agility towards previously unseen domains.
alternates between the two sub-problems below: where superscript i is the instance index, and D(·) is a discriminator the domain-encoder E domain (·) tries to confuse. We first train the discriminator D(·) with the labeled data in the source domain. For the data in the target domain, we assign them pseudo-labels by the classifier Φ(E class (·)) we have trained earlier. The learned domain encoder E domain (·) is class-confusing due to z i random , a random label uniformly chosen in the label space. As the classifier D(·) is trained, the first sub-problem essentially learns the domain-encoder such that it classifies the input x i into a random class z i random . Algorithm 2 details our domain disentanglement process.

Curriculum Domain Adaptation
We rank all the instances in the compound target domain according to their distances to the source domain, to be used for curriculum domain adaptation [56]. We compute the domain gap between a target instance x t and the source domain {x m s } as their mean distance in the domain feature space: . We train the network in stages, a few epochs at a time, gradually recruiting more instances that are increasingly far from the source domain. At each stage of the curriculum Algorithm 1 Domain Disentanglement.
Input: The class encoder E class (·) and classifier Φ have been trained using source-domain data, Deccoder(·): the decoder, C: the number of classes, γ: a constant.
Compute adversarial loss: end for learning, we minimize two losses: One is the cross-entropy loss defined over the labeled source domain, and the other is the domain-confusion loss [48] computed between the source domain and the currently covered target instances. Figure 4 (c) illustrates a curriculum in our experiments.

Memory Module for Open Domains
Existing domain adaptation methods often use the features v direct extracted directly from the input for adaptation. When the input comes from a new domain that significantly differs from the seen domains during training, this representation becomes inadequate and could fool the classifier. We propose a memory module to enhance our model; It allows knowledge transfer from the source domain so that the network can dynamically balance the input-conveyed information and the memory-transferred knowledge for more classification agility towards previously unseen domains.  Class Memory M . We design a memory module M to store the class information from the source domain. Inspired by [45,36,28] on prototype analysis, we also use class centroids {c k } K k=1 to construct our memory M , where K is the number of object classes.
Enhancer v enhance . For each input instance, we build an enhancer to augment its direct representation v direct with knowledge in the memory about the source domain: is a softmax function. We add this enhancer to the direct representation v direct , weighted by a domain indicator. Domain Indicator e domain . With open domains, the network must dynamically calibrate how much knowledge to transfer from the source domain and how much to rely on the direct representation v direct of the input. Intuitively, the larger domain gap between an input x and the source domain, the more weight on the memory feature. We design a domain indicator for such domain awareness: e domain = T (E domain (x)), where T (·) is a lightweight network with the tanh activation functions and E domain (·) is the domain encoder we have learned earlier.
Source-Enhanced Representation v transf er . Our final representation of the input is a dynamically balanced version between the direct image feature and the memory enhanced feature: which transfers class-discriminative knowledge from the labeled source domain to the input in a domain-aware manner.
Operator ⊗ is element-wise multiplication. Adopting cosine classifiers [27,10], we 2 -normalize this representation before sending it to the softmax classification layer. All of these choices help cope with domain mismatch when the input is significantly different from the source domain.

Experiments
Datasets. To facilitate a comprehensive evaluation on various tasks (i.e., classification, segmentation, and navigation), we carefully design four open compound domain adaptation (OCDA) benchmarks: C-Digits, C-Faces, C-Driving, and C-Mazes, respectively.   Network Architectures. To make a fair comparison with previous works [48,9,39], the modified LeNet-5 [21] and ResNet-18 [15] are used as the backbone networks for C-Digits and C-Faces, respectively. Following [47,58,35], a pre-trained VGG-16 [44] is the backbone network for C-Driving. We additionally test our approach on reinforcement learning using ResNet-18 following [18]. Evaluation Metrics. The C-digits performance is measured by the digit classification accuracy, and the C-Faces performance is measured by the facial expression classification accuracy. The C-Driving performance is measured by the standard mIOU, and the C-Mazes performance is measured by the average successful rate in 300 steps. We evaluate the performance of each method with five runs and report both the mean and standard deviation. Moreover, we report both results of individual domains and the averaged results for a comprehensive analysis. Comparison Methods.
For classification tasks, we choose for comparison state-of-the-art methods in both conventional unsupervised domain adaptation (ADDA [48], JAN [30], MCD [42]) and the recent multi-target domain adaptation methods (MTDA [9], BTDA [5], DADA [39]). Since MTDA [9], BTDA [5] and DADA [39] are the most related to our work, we directly contrast our results to the numbers reported in their papers. For the segmentation task, we compare with three state-of-the-art methods, AdaptSeg [47], CBST [58], IBN-Net [35] and PyCDA [26]. For the reinforcement learning task, we benchmark with MTL, MLP [18] and SynPo [18], a representative work for adaptation across environments. We apply these methods to the same backbone networks as ours for a fair comparison.

Ablation Study
Effectiveness of the Domain-Focused Factors Disentanglement. Here we verify that the domain-focused factors disentanglement helps discover the latent structures in the compound target domain. It is probed by the domain identification rate within the k-nearest neighbors found by different encodings. Figure 5 (b) shows that features produced by our disentanglement have a much higher identification rate (∼95%) than the counterparts without disentanglement (∼65%).
Effectiveness of the Curriculum Domain Adaptation. Figure 5 (a) also reveals that, in the compound domain, the curriculum training contributes to the performance on USPS more than MNIST and MNITS-M. On the other hand, we can observe from Figure 4 and Table 2 that USPS is the furthest target domain from the source domain SVHN. It implies that curriculum domain adaptation makes it easy to adapt to the distant target domains through an easy-to-hard adaptation schedule.  Table 4: Performance on the C-Driving (left) and C-Mazes benchmarks (right). "SynPo+Aug." indicates that we equip SynPo with proper color augmentation/randomization during training. Visual illustrations of both datasets are in Figure 6. Effectiveness of Memory-Enhanced Representations.
Recall that the memory-enhanced representations consist of two main components: the enhancer coming from the memory and the domain indicator. From Figure 5 (a), we observe that the class enhancer leads to large improvements on all target domains. It is because the enhancer from the memory transfers useful semantic concepts to the input of any domain. Another observation is that the domain indicator is the most effective on the open domain ("SynNum"), because it helps dynamically calibrate the representations by leveraging domain relations ( Figure 5 (c)). C-Driving. We compare with the state-of-the-art semantic segmentation adaptation methods such as AdaptSeg [47], CBST [58], and IBN-Net [35]. All methods are tested under real-world driving scenarios in the BDD100K dataset [53].

C-Digits.
We can see that our approach has clear advantages on both the compound domain (1.1% gains) and the open domain (2.4% gains) as shown in Table 4 (left). We show detailed per-class accuracies in the appendices. The qualitative comparisions are shown in Figure 6 (a). C-Mazes. To directly compare with SynPo [18], we also evaluate on the GridWorld environments they provided. The task in this benchmark is to learn navigation policies that can successfully collect all the treasures in the given mazes. Existing reinforcement learning methods suffer from environmental changes, which we simulate as the appearances of the mazes here. The final results are listed in Table 4 (right). Our approach transfers visual knowledge among navigation experiences and achieves more than 30% improvements over the prior arts.  Figure 8 presents the t-SNE visualization comparison between the obtained embeddings of JAN [30], MCD [42], and our approach.

Summary
We formalize a more realistic topic called open compound domain adaptation for domain-robust learning. We propose a novel model which includes a self-organizing curriculum domain adaptation to bootstrap generalization and a memory enhanced feature representation to build agility towards open domains. We develop several benchmarks on classification, recognition, segmentation, and reinforcement learning and demonstrate the effectiveness of our model.

Appendices
In this supplementary material, we provide details omitted in the main text including: • Section A: relation to other DA problems (Sec. 2 "Related Works" of the main paper.) • Section B: more methodology details (Sec. 3 "Our Approach" of the main paper.) • Section C: detailed experimental setup (Sec. 4 "Experiments" of the main paper.) • Section D: additional comparison results (Sec. 4.2 "Comparison Results" of the main paper.) • Section E: additional visualization of our approach (Sec. 4.3 "Further Analysis" of the main paper.)

A. Relation to Other DA Problems
Human vision system shows remarkable generalization ability to see clear across many different domains, such as in foggy and rainy days. Computer vision system, on the other hand, has long been haunted by this domain shift issue. Several sub-fields in domain adaptation have been studies to mitigate this challenge. Open/Partial Set Domain Adaptation. Another route of research aims to tackle the category sharing/unsharing issues between source and target domain, namely open set [37,43] and partial set [55,3] domain adaptation. They assume that the target domain contains either (1) new categories that don't appear in source domain; or (2) only a subset of categories that appear in source domain. Both settings concern the "openness" of categories. Instead, in this work we investigate the "openness" of domains, i.e. we assume there are unknown domains existing that are absent in the training phase.

B. More Methodology Details
Notation Summary. We summarize the notations used in the paper in Table 5.

Details of Class and Domain Manifold Disentanglement.
The detailed class and domain manifold disentanglement algorithm is shown in Algorithm 2. Time Complexity. Our approach introduces negligible computational overhead (1.3%) to the standard deep networks, such as VGG [44] and ResNet [15], since only a lightweight memory module is inserted during inference. Methodology Highlight. Our main methodology contribution is the entire neural architecture that can address the complexity of compound domains during training and handle the unseen domains during testing, as depicted in Figure 9. network that generates e domain v transf er source-enhanced representation Algorithm 2 Disentangling training. E class (·) and Φ has been trained using source-domain data, Deccoder(·): the decoder, C: number of classes, γ: a constant.
Compute pseudo label y pseudo ← Φ (E class (x)). Update the discriminator D with: ∇ θ D j y j pseudo log D E domain x j . Prepare random label z random ∼ unif orm{0, 1, ..., C − 1}, and convert it to one-hot vector y random .
GTA5 is a virtual street view dataset generated from Grand Theft Auto V (GTA5). It has 24966 images of resolution 1914 × 1052. The domain categories and statistics of the BDD100K dataset are provided in Table 7. C-Mazes. This is a reinforcement learning dataset which is consist of mazes with different colors. The mazes are generated following [18], where agent is asked to collect treasure spots from the mazes by different orders. For simplicity, we only use one scene (i.e. maze with one topology) and two tasks (i.e. two different treasure collection orders) for the experiments. We randomly generated 5 combinations of colors to construct the domains (which are illustrated in Figure 11). The colors of agent and five treasures are the same across the domain. We use Domain 0 as the source domain, and the rest of the 4 domains are target domains.

C.2. Training Details
C-Digits. The backbone model for this experiment is a LeNet-5 [21], following the setups in [17]. There are two stages in the training process.
(1) In the first stage, we train the network with discriminative centroid loss for 100 epochs as a warm start for the memory-enhanced deep neural network. (2) In the second stage, we further finetune the networks with curriculum sampling and memory modules on top of the backbone network, where a domain adversarial loss [48] is incorporated and the weights learned in the first stage are copied to two identical but independent networks (source network and target network). Only weights of the target network are updated during this stage. Centroids of each class (i.e. constituting elements in the class memory) are calculated in the beginning of this stage, and the classifiers are reinitialized. The model is trained without the discriminative centroid loss in stage 2. Some major hyper-parameters can be found in Table 8. C-Faces. Experiments on the C-Faces dataset are similar to experiments on the C-Digits dataset. However, the backbone model is ResNet18 [15] with random initialization, instead of a LeNet-5. Some major hyper-parameters can also be found in Table 8.  [20] and 0.005 initial learning rate for the discriminator.
In our method, we use dynamic transferable embedding and curriculum training in addition to adversarial adaptation [47]. The visual memory here also comprises of a set of class centroids, which is an aggregation of local features belonging to the same category. Inspired by [58], the curriculum learning procedure is further designed to include the averaged probability confidence of each image as a guidance. Specifically, the samples that are easier, i.e., have higher confidence, are firstly fed into the model for adaptation. Since domain encoder is not accessible here, we only use class enhancer for dynamic transferable embedding.

C-Mazes.
The experiments for C-Mazes are also conducted in two-stages. In the first stage, We follow the setups in [18] to train a randomly initialized ResNet-18 policy network. We only use one single topology and two different tasks for the experiments. The total episodes is set to 16000. The initial learning rate is set to 0.001. Then in the second stage, we use the pre-trained model to calculate state feature centers for each actions and use memory module to finetune the model.

D. Additional Results
C-Digits. We have experimented with mnist, mnist-m or usps as the source domain. On average, the performance gain of our approach is 8.1% over the baseline method JAN [30] and 8.9% over MCD [42]; Likewise, the average performance gain with Multi-PIE as the source domain is 36.5% and 14.4% over the baselines. C-Driving. Figure 13 shows example results on the GTA5 and BDD100K dataset. Our method produces more accurate segmentation results on the compound domains and the open domain compared to 'source only' and AdaptSeg [47]. Per-category results are provided in Table 9, and ablation study of dynamic transferable embedding and curriculum training are shown in Figure 12. Office-Home [49]. We had some preliminary results on Office-Home [49], where our approach outperforms baseline methods (JAN [30] and MCD [42]) by 15.3% and 7.9%, respectively.
E. More Visualizations t-SNE Visualization. Here we show t-SNE visualizations of the learned dynamic transferable embedding on the C-Digits, C-Faces, and C-Mazes testing data (Figure 14 -16), among various methods. The dynamic transferable embedding on the C-Mazes benchmark are state features of each actions. Our approach generally learns a more discriminative feature space thanks to the proposed disentanglement and memory modules. Confusion Matrices. Here we show visualizations of class confusion matrices on the C-Digits and C-Faces testing data in Figure 17 and Figure 18. Compared to MCD [42] and JAN [30], our approach performs better on the discriminative accuracies of each class.        [42], JAN [30], and our approach. Figure 18: Confusion matrices visualizations on the C-Faces benchmark for MCD [42], JAN [30], and our approach.