{'fraction_non_alphanumeric': 0.02590112665891504, 'fraction_numerical': 0.012870374082835492, 'mean_word_length': 4.80451477775192, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Unsupervised Domain Adaptation Model based on Dual-module Adversarial Training

In this paper, we propose a dual-module network architecture that employs a domain discriminative feature module to encourage the domain invariant feature module to learn more domain invariant features. The proposed architecture can be applied to any model that utilizes domain invariant features for unsupervised domain adaptation to improve its ability to extract domain invariant features. We conduct experiments with the Domain-Adversarial Training of Neural Networks (DANN) model as a representative algorithm. In the training process, we supply the same input to the two modules and then extract their feature distribution and prediction results respectively. We propose a discrepancy loss to find the discrepancy of the prediction results and the feature distribution between the two modules. Through the adversarial training by maximizing the loss of their feature distribution and minimizing the discrepancy of their prediction results, the two modules are encouraged to learn more domain discriminative and domain invariant features respectively. Extensive comparative evaluations are conducted and the proposed approach outperforms the state-of-the-art in most unsupervised domain adaptation tasks.


Introduction
In recent years, deep convolutional neural networks (CNNs) have achieved huge success in many computer vision applications, such as image classification [6,28], object detection [27,37], tracking [62], segmentation [20,43], image generation [58], and crowd counting [48]. A lot of powerful network architectures have been proposed for efficient feature extraction, representation, and optimization [19,59,63]. However, most of these models need to be trained via supervised learning from a large collection of reliable training data.
In many practical application scenarios, we may not have or it is too expensive to obtain a lot of reliable labeled data, and labeling data correctly and accurately is usually more difficult than collecting data. For example, in medical image processing, labeling data correctly requires people to have sufficient professional training and domain knowledge. Therefore, how to make full use of unlabeled data becomes a more and more attractive topic in the computer vision field. An effective method to solve this kind of problem is to use unsupervised domain adaptation (UDA).
In unsupervised domain adaptation, we assume that there are two data sets.
One is an unlabeled data set from the target task, called the target domain.
The other data set is a labeled data set from the source task, called the source domain. There are some similarities between the two data sets, however, there are still some differences between them. For example, cartoon characters and real human beings have some commonalities, but we can also clearly distinguish the differences between them. When we train a model with the source domain data set, and then apply the trained model to the source task and the target task respectively, the performance for the target domain is usually significantly lower than that for the source domain. This is due to the domain shift between the source domain and the target domain. The extent of these reductions depends on the domain shift. If the domain shift is larger, the performance loss for the target task becomes greater. To solve this problem, we need to know how to align the source domain with the target domain and reduce the shift between the two domains.

With the advent of gradient reversal layers (GRL) from Domain-Adversarial
Training of Neural Networks (DANN) [12], more and more people realize that adversarial training has a significant effect on aligning the feature of the source and target domains. DANN [12] extracts the global invariant features by training a domain discriminator to fool the feature extractor. The domain discriminator is a component composed of several fully connected layers. Its function is to distinguish the input data from the source domain or the target domain. If the discriminator could not recognize the extracted feature map, it means that the extracted feature map comes from the common space of these two domains.
The global invariant features are from this space, and DANN utilizes a classifier for adversarial training to ensure the effectiveness of the features learned by the network. In addition to the adversarial training method based on GRL, there are many other adversarial training methods based on generative adversarial nets (GANs) [15,32,52,22,17]. Most of them have one thing in common, i.e., they adapt to the target domain by learning global invariant features.
Motivation. Taking DANN [12] as an example, although it has the advantage to use a discriminator to encourage the model to learn invariant features, there is also a bottleneck. When we only rely on the discriminator to control the extraction ability of the domain invariant features, the conditions to extract domain invariant features will become very limited. When the extracted domain invariant features are strong enough to fool the discriminator, it will become difficult for the model to further improve its ability to extract the domain invariant features. In addition, when we only use the source domain labeled data to adjust the classifier, the extracted features will be more biased towards the source domain, which further limits its performance on the target task. This challenge also exists in other methods that employ the domain discriminator to extract the domain invariant features.
Contributions. In order to further improve the extraction ability of domain invariant features, we propose a dual-module architecture to solve this challenge. As shown in Figure 1, The proposed network is composed of a discriminative feature learning module and a domain invariant feature learning module, and the domain discriminative feature module is employed to encourage a domain invariant feature module to learn more domain invariant features.
We take DANN [12] as an example to implement our model. We also employ the Maximum Classifier Discrepancy (MCD) [47] to learn the domain discriminative features from the target domain so that the classifier is not only affected by the domain discriminative features of the source domain.
In summary, the main contributions of this paper include: 1. We propose a novel dual-module network architecture to promote the learning of domain invariant features; 2. We propose a new adversarial loss to maximize the discrepancy of the feature distributions while minimizing the discrepancy of the prediction results; 3. We propose to maximize the classifier discrepancy to ameliorate the training imbalance issue of previous unsupervised domain adaptation methods.
We evaluated our proposed method on four popular benchmark datasets and compared its performance with previous work. The proposed structure outperforms the state-of-the-art by a large margin on unsupervised domain adaptation of digit classification and object recognition. The source code of the proposed model can be accessed at the author's website 1 .

Related Work
Domain adaptation could be taken as a subset of transfer learning [41], which is a commonly used technique in many computer vision tasks to improve the generalization ability of a model trained on a single domain. In this section, we describe some existing domain adaptation methods.
Learn domain invariant features. Recently, [8] explored what enables the contrastive prediction tasks to learn useful representations. [5] learns the semantic labels in a supervised fashion, and broadens its understanding of the data by learning from self-supervised signals how to solve a jigsaw puzzle on the same images. DICD [29] proposes to learn domain invariant features and class discriminative features simultaneously by a low-dimension latent feature space so that the samples with the same category are close to each other while the samples with different categories are away from each other.
Distance-based methods. Aligning the distribution between the source domain and the target domain is a very common method in solving unsupervised domain adaptation problems. Maximum Mean discrepancy (MMD) [16,35,54,33] is a method of measuring the difference of two distributions. DAN [33] explored the multi-core version of MMD to define the distance between two distributions. JAN [35]  To solve the problem of unbalanced datasets, Deep Asymmetric Transfer Network (DATN) [56] proposed to learn a transfer function from the target domain to the source domain and meanwhile adapting the source domain classifier with more discriminative power to the target domain. DeepCORAL [51] builds a specific deep neural network by aligning the distribution of second-order statistics to limit the invariant domain of the top layer. [7] proposed a Higher-order Moment Matching (HoMM) method to minimize the domain discrepancy.
Adversarial methods. Adversarial training is another very effective method to transfer domain information. Inspired by the work of gradient reversal layer [12], a group of domain adaptation methods has been proposed based on adversarial learning. RevGrad [12] proposed to learn the global invariant feature by using a discriminator that is used to reduce the discriminative features in the domain. Deep Reconstruction-Classification Networks (DRCN) [13] jointly learned a shared encoding representation for supervised classification of the labeled source data, and unsupervised reconstruction of the unlabeled target data.
[3] extracted image representations that are partitioned into two subspaces. Adversarial Discriminative Domain Adaptation (ADDA) [53] trained two feature extractors for the source and target domains respectively, to generate embeddings to fool the discriminator.

Proposed Method
In this paper, we consider the unsupervised domain adaptation problem.
K is the number of categories. We assume that the source sample x s belongs to the source distribution P s , and the target sample x t belongs to the target distribution P t , where P s = P t . Our goal is to train a classifier f θ (x) that can minimize the target risk t = E x∈Dt [f θ (x) = y t ], where f θ (x) represents the output of the deep neural network, and θ represents the model parameters to be learned.
In the following subsections, we will present the main idea of the proposed strategy for unsupervised domain adaptation, the network architecture, and the training steps of the network.

Main Idea
In the method that employs the domain discriminator to help the feature  of DANN, only the source data is used to adjust the classifier. Therefore, we adopt Maximum Classifier Distance (MCD) [47] to learn the domain discriminative features from the target domain after the model reaches its upper limit to further improve its performance.
Dual-module architecture. In the dual-module architecture, we need to build two modules with completely different functions. As shown in Figure 2, We call the module that can learn the domain invariant features as M1, and the module that can learn domain discriminative features as M2. Different from [10] which employs coupled neural networks for the source domain and target domain respectively, both modules in our model receive the data from the source domain and the target domain since each module has a discriminator. For the module M1, we can use any algorithm that uses a discriminator to learn domain invariant features. In our experiment, we use DANN [12] as the module M1, since it has a clear and concise structure. For the module M2, the easiest way is to employ the source-only method. As the number of training increases, the ability of the feature extractor to extract domain discriminative features will be greatly improved. In other words, among the features extracted by M2, the domain discriminative features will have a larger ratio than the domain invariant features.
In our experiments, our M2 also follows the same DANN structure. The only difference is that we remove the gradient reversal layer for adversarial training, so the discriminator will make the learned features more discriminative.
As a result, when we input the same data into the two modules, the feature distribution from M1 will be based on domain invariant features, and the feature distribution from M2 will be based on domain discriminative features. We designed an adversarial loss for training this dual-module architecture. In the adversarial loss, we expand the discrepancy in feature distribution between the two modules. At the same time, we narrow the discrepancy in the prediction between the two modules. We will provide more details about this in Section 3.4.
Target domain discriminative feature. Maximum Classifier Discrepancy (MCD) [47] is to make the model learn the decision boundary of a specific task by maximizing the prediction difference of the two classifiers. Since the two classifiers are iterated under the same conditions, their prediction results are very similar. However, for some data that is easy to be confused, the results given by the two classifiers may be different. Therefore, if the information captured by the feature extractor is more discriminative, the features that determine the decision boundary will also be larger. We follow this idea and use MCD to optimize our classifier, since the classifier of DANN is only adjusted by the source domain. We add a classifier to each of the two modules, which means that there are two classifiers with the same structure in each module.
We assign these two classifiers to random parameters and train them together.
We make our module learn the domain discriminative features from the target domain by learning the decision boundary. This can make up for the lack of the domain discriminative features only from the source domain.

Network Structure
In this section, we will elaborate on the proposed network structure in detail.
As shown in Figure 2, our network employs a dual-module structure. M1 is the domain invariant feature module, and M2 is the domain discriminative feature module. M1 is composed of a feature extractor G1, a discriminator D1, two classifiers C1 and C2, and a Linear transformation layer T1. M2 consists of a feature extractor G2, a discriminator D2, two classifiers C3 and C4, and a Linear transformation layer T2. Since the loss of our dual-module architecture needs to calculate the discrepancy of the two feature distributions, we need an independent linear transformation layer to convert the feature map into a feature distribution. For each module, We embed a linear transformation layer after the feature extractor. For the linear transformation layer, we set its input and output sizes as that of the feature extractor, so as to ensure that there is no information loss in this process.
In addition, the domain discriminators in these two modules have completely different functions. The discriminator D1 plays the same role as the discriminator in DANN to fuse the two domains. However, the discriminator D2 is employed to separate the two domains. Therefore, M1 learns domain invariant features, while M2 learns the domain discriminative features. Please note that the sub-components used in our two modules have exactly the same structure.
For example, G1 and G2 have the same structure, D1 and D2 are the same, and C1, C2, C3, and C4 are the same.

Discrepancy Loss
In this study, we follow the discrepancy loss in [47]. We use the absolute value of the difference between the probability outputs of the two classifiers as the discrepancy loss: where p 1 and p 2 are the probability outputs of the two classifiers respectively, which are the prediction scores for all the categories, and K is the number of categories, and p 1 k and p 2 k are the specific values of their k th category.

Training Steps
Step 1: Our model learns the decision boundary by using MCD [47]. Learning the decision boundary is essential to learning the discriminative feature. The Step 2: In this step, we continue to train the two modules separately.
Taking the M1 module as an example, we use the adversarial loss of DANN for training. In other words, we train the model according to the training method from the original algorithm. After this step, the two modules begin to have some differences. This step is necessary for the proposed dual-module structure. For the algorithms that do not use MCD for pre-processing, this will be the first step in the entire training process.
For M1, we conduct adversarial training through gradient reversal layer (grl) to learn the domain invariant features.
where L M 1 is the total loss of the whole M1 module, L C is the total loss of two classifiers, L C1 , L C2 , and L D1 are the cross-entropy loss for the classifiers and discriminator.
M2 does not have the gradient reversal layer, so the discriminator will prompt the feature extractor to learn the domain discriminative features.
where L M 2 is the total loss of the whole M2 module, L C is the total loss of the two classifiers, L C3 , L C4 , and L D2 are the cross-entropy loss for the classifiers and discriminator.
Step 3: In this step, we conduct an adversarial loss function L for our two modules. The two modules have separate parameters and they do not share parameters with each other. Thus, we intend to maximize the discrepancy of the features extracted by the feature extractors while minimizing the difference of the predicted classification results so that the feature extractors could concentrate on invariant features and discriminative features respectively, while the predicted results would not have much difference. We input the same set of data into the two modules and extract the output from the transformation layer T and the classifier C. We use the linear transformation layer to convert the feature maps generated by the feature extractors into feature distributions, and calculate the discrepancy between the two modules, as illustrated in Figure   2. We use a gradient reversal layer (grl) to maximize the discrepancy of the feature distributions of the two modules.
Although there are two classifiers in each module, we only utilize C1 (from M1) and C3 (from M2) to calculate the discrepancy between the predictions of the two modules. At this time, the discrepancy of the predicted results between the two modules is minimized so that, although the feature extractors in the two modules focus on invariant features and discriminative features respectively, the predicted results would not have too much discrepancy.
Our adversarial loss function is to play a Min-Max game with these two discrepancies.
where L is the total loss, dis() is the discrepancy loss. t s 1 means the output is from T1 and the input is from the source domain, and t s 2 means the output from T2 and input from the source domain. t t 1 means the output from T1 and input from the target domain, and t t 2 means the output from T2 and input from the target domain. c s 1 means the probability output from C1 and takes input from the source domain, and c t 1 means the probability output from C1 and takes input from the target domain. c s 3 means the probability output from C3 and takes input from the source domain, and c t 3 means the probability output from C3 and takes input from the target domain.

Experiments
We conducted extensive experiments to evaluate the proposed architecture and the effect of different components in the architecture. We conduct our experiments on three digits datasets, two traffic sign datasets, and one object classification dataset. In the following, OURS mentioned in the results refers to the case where only the dual-module structure is used. MCD+DANN refers to the case where MCD is directly employed to solve the imbalance problem in DANN. OURS+1M refers to the case where MCD is only used by the M1 module. OURS+2M refers to the case where MCD is used by both M1 and M2 modules. We also compare the performance with DANN and MCD as the baselines.

MNIST:
The dataset contains images of digits 0 to 9 in different styles. It is composed of 60, 000 training and 10, 000 testing images.
USPS: This is also a digit dataset with 7, 291 training and 2, 007 testing images. We evaluate the unsupervised domain adaptation model on the following four transfer scenarios: During the experiments, we employ the CNN architecture and the input size used in [47]. We used mini-batch stochastic gradient descent (SGD) to optimize our model and set the learning rate at 0.002 in all experiments. We  Table 1: The performance on digit classification and sign classification. We report the mean and the standard deviation of the accuracy obtained over 5 trials.
follow DANN [12] and employ the SGD training schedule for the part of learning domain invariant feature: the learning rate adjusted by η p = η0 (1+αp) β , where p denotes the process of training iterations that is normalized in [0, 1], and we set η 0 = 0.002, α = 10, and β = 0.75; the hyper-parameter λ is initialized at 0 and is gradually increased to 1 by λ p = In addition, our model outperforms the state-of-the-art methods on all tasks.

Experiments on VisDA Classification Dataset
We further evaluate our model on the large VisDA-2017 dataset [44]. The VisDA-2017 image classification is a 12-class domain adaptation dataset used to evaluate the adaptation from synthetic-object to real-object images. The source domain consists of 152,397 synthetic images, where 3D CAD models are rendered from various conditions. The target domain consists of 55,388 real images taken from the MS-COCO dataset [30].
In this experiment, we employ Resnet-18 [19] as our feature extractor, and the parameters are adopted from the ImageNet pre-trained model. The pretrained model of our Resnet-18 comes from Pytorch [42] and all experimental implementations are based on Pytorch.
The input images are of the size 224 × 224. First, we resize the input image to 256, and then crop the image to 224 × 224 in the center. When we train the model using only the source domain, we just modify the output size of the original last fully connected layer to a size that conforms to VisDA-2017 [44]. In other tasks, we utilize a three-layer fully connected layer structure to replace the one-layer fully connected layer structure of the original classifier. For algorithms that require a discriminator, we employ a discriminator with a three-layer fully connected layer structure. In order to eliminate the interference factors, except for the source only, all other algorithms use the same classifier and the same discriminator. We uniformly use SGD as the optimizer for training, and use 1 × 10 −3 for the learning rate of all methods. We use 64 as the batch size for training.
The results on VisDA-2017 are shown in Table 2. We can see that our model achieves an accuracy much higher than previous methods. In addition, our method performs better than the source only model in all classes, whereas MCD and DANN perform worse than the source only model in some classes such as train, motorcycle, and car. Same as the last experiment, OURS+2M is the top-performing variant. On average, we obtain an improvement of 6.1% compared with MCD, and 13.3% compared with DANN. Our model also achieves the best performance in eight out of the twelve categories in this experiment.

Ablation Study
We conducted ablation studies using the digital and traffic sign datasets with the same unsupervised domain adaptation setting as Section 4.1. The proposed algorithm has two modules, and each module has two partial components, namely dif-module, ddf-module, dif-MCD and ddf-MCD. The dif-module refers to the component used to learn domain invariant features in the model. This is a necessary module in the algorithm and also in our baseline.   As shown in Table 3, the performance of dual-module adversarial training has a significant improvement over using a single domain adaptation method.
There are two most intuitive examples. (i) The performance of DAN N with dual-module adversarial training is 18.5% higher than the one without dual-

Visualization
In Figure 3, we use T-SNE [55] to visualize the models we trained. We choose the task that transfer SYN SIGNS [39] to GTSRB [50]. SYN SIGNS is the source domain, and GTSRB is the target domain. After training, we select 2000 data for visualization, where 1000 images from the source domain and 1000 images from the target domain. It can be clearly seen from the result that, compared to the Source Only method, our proposed model has a significant effect on reducing the domain shift, especially for the OURS+M2 variant.

Conclusion and Future Work
In this paper, we have proposed a dual-module network architecture that can We are currently working on the applications for other UDA tasks like object detection and segmentation.