{'fraction_non_alphanumeric': 0.04192859506120853, 'fraction_numerical': 0.013625289499804493, 'mean_word_length': 4.466622821440316, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Domain Adaptation via Maximizing Surrogate Mutual Information

Unsupervised domain adaptation (UDA) aims to predict unlabeled data from target domain with access to labeled data from the source domain. In this work, we propose a novel framework called SIDA (Surrogate Mutual Information Maximization Domain Adaptation) with strong theoretical guarantees. To be specific, SIDA implements adaptation by maximizing mutual information (MI) between features. In the framework, a surrogate joint distribution models the underlying joint distribution of the unlabeled target domain. Our theoretical analysis validates SIDA by bounding the expected risk on target domain with MI and surrogate distribution bias. Experiments show that our approach is comparable with state-of-the-art unsupervised adaptation methods on standard UDA tasks.


Introduction
Inspired by human beings' ability to transfer knowledge across domains and tasks, transfer learning is proposed to leverage knowledge from source domain and task to improve performance on target domain and task. However, in practice, labeled data are often limited on target domains. To address such situation, unsupervised domain adaptation (UDA), a category of transfer learning methods [Long et al., 2015;Long et al., 2017b;, attempts to enhance knowledge transfer from labeled source domain to target domain by leveraging unlabeled target domain data.
Most previous work is based on the data shift assumption, i.e., the label space maintains the same across domains, but the data distribution conditioned on labels varies. Under this hypothesis, domain alignment and class-level method are used to improve generalization across source and target feature distributions. Domain alignment minimizes the discrepancy between the feature distributions of two domains [Long et al., 2015;Long et al., 2017b], while class-level methods work on conditional distributions. Conditional alignment aligns conditional distributions and use pseudo-labels to estimate conditional distribution on target domain Li et al., 2020c;Chen et al., 2020a]. However, the conditional distributions from different categories tend to mix together, leading to performance drop.
Contrastive learning based methods resolve this issue by discriminating features from different classes , but still face the problem of pseudo-label precision. In addition, most of the class-level methods lack solid theoretical explanations for the relationship between cross domain generalization and their objectives. Some works Xie et al., 2018] yield some intuition for conditional alignment and contrastive learning, but the relation between their training objectives and cross-domain error remains unclear.
In this work, we aim to address the generalization problem in domain adaptation from an information theory perspective. In failed case of domain adaptation, as shown in Figure  1, features from the same class do not represent each other well and this inspires us to use mutual information to reduce this confusion. Our motivation is to find more representative features for both domains by maximizing mutual information between features of the same class (on both source and target domains). Therefore, if our classifier can accurately predict features on source domain, then it would also function well on target domains where features share enough information with the source features.
Based on the above motivation, we propose Surrogate Information Domain Adaptation (SIDA), a general domain adaptation framework with strong theoretical guarantees. SIDA achieves adaptation by maximizing the mutual information (MI) between features within the same class, which improves the generalization of the model to the target domain. Furthermore, a surrogate distribution is constructed to approximate the unlabeled target distribution, which improves flexibility for selecting data and assists MI estimation. Also, our theoretical analyses directly establish a bound between MI of features and target expected risk, giving a proof that our model can improve generalization across domain.
Our novelties and contributions are summarized as follows: • We propose a novel framework to achieve domain adaptation by maximizing surrogate MI.
• We establish an expected risk upper bound based on feature MI and surrogate distribution bias for UDA. This provides theoretical guarantee for our framework.
• Experiment results on three challenging benchmarks demonstrate that our method performs favorably against state-of-art class-level UDA models. Figure 1: Our motivation is that higher intra-class feature mutual information implies better generalization ability across domain. As shown, unsuccessful domain adaptation implies lower intra-class feature mutual information. This is reflected in the fact that features from the same class do not represent each other well because of confusion with features from other classes.

Related Work
Domain Adaptation Prior works are based on two major assumptions: (1) the label shift hypothesis, where the label distribution changes, and (2) a more common data shift hypothesis where we only study the shift in conditional distribution under the premise that the label distribution is fixed. Our work focuses on the data shift hypothesis, and previous work following this line can be divided into two major categories: domain alignment methods which align marginal distributions, and class-level methods addressing the alignment of conditional distributions. Domain alignment methods minimize the difference between feature distributions of source and target domains with various metrics, e.g. maximum mean discrepancy (MMD) [Long et al., 2015], JS divergence  estimated by adversarial discriminator, Wasserstein metric and others. Maximum Mean Discrepancy (MMD) is applied to measure the discrepancy in marginal distributions [Long et al., 2015;Long et al., 2017b]. Adversarial domain adaptation plays a mini-max game to learn domain-invariant features Li et al., 2020b].
Class-level methods align the conditional distribution based on pseudo-labels Chen et al., 2020a;Li et al., 2020a;Xu et al., 2019]. Conditional alignment methods  minimize the discrepancy between conditional distributions. In class-level methods, conditional distributions are assigned by pseudo-labels. The accuracy of pseudo-labels greatly influences performance and later works construct more accurate pseudo-labels [Chen et al., 2020a]. However, the major problem with this method is that error in conditional alignment leads to distribution overlap of features from different class, resulting in low discriminability on target domain. Contrastive learning addresses this problem by maximizing the discrepancy between different classes Li et al., 2020a]. However, the performance of contrastive learning also relies on pseudo-labeling.
In addition, previous class-level works provide weak theoretical support for cross-domain generalization. Prior works mainly focus on domain alignment . Some works Xie et al., 2018] consider optimal classification on both domains, and yield some intuitive explanation for conditional alignment and contrastive learning, but the relation between their objective function and theoretical cross-domain error remains unclear.
Information Maximization Principle Recently, mutual information maximization (InfoMax) for representation learning has attracted lots of attention [Chen et al., 2020b;Khosla et al., 2020]. The intuition is that two features belonging to different classes should be discriminable while features of the same class should resemble each other. The InfoMax principle provides a general framework for learning informative representations, and provides consistent boosts in various downstream tasks.
We facilitate domain adaptation with MI maximization, i.e. maximizing the MI between features of the same class. Some works solve domain adaptation problem via information theoretical methods [Thota and Leontidis, 2021;Park et al., 2020], which maximize MI using InfoNCE estimation . As far as we know, we are the first to provide theoretical guarantee for the target domain expected risk based on MI. Compared with InfoNCE, the variational lower bound of MI we use is tighter [Poole et al., 2019]. We also construct a surrogate distribution as a substitute for unlabeled target domain, which is more suitable for MI estimation.

Notations and Problem Setting
Let X be the data space and Y be the label space. In UDA, there is a source distribution P S (X, Y ) and a target distribution P T (X, Y ) on X × Y. Note that distributions are also referred to as domains in UDA. Our work is based on the data shift hypothesis, which assumes P S (X, Y ) and P T (X, Y ) satisfy the following properties: P T (Y ) = P S (Y ) and P T (X|Y ) = P S (X|Y ) .
In our work, we focus on classification tasks. Under this setting, an algorithm has access to n S labeled sam- , and outputs a hypothesis composed of an encoder G and a classifier F . Let Z be the feature space. The encoder maps data to feature space, denoted by G : X → Z. Then the classifier maps the feature to a corresponding class, F : Z → Y.
For brevity, given encoder G and data-label distribution P (X, Y ), denote the distribution of G-encoded feature and label by P G , i.e. P G (z, y) = P (x = G −1 (z), y). Let F be a hypothesis, and P be the distribution of feature and label. The expected risk of a F w.r.t. P is denoted as (1) where δ F (z) (y) equals to 1 if y = F (z) and equals 0 in else cases. Our objective is to minimize the expected risk of F on target feature distribution encoded by G, 4 Methodology

Overview
In UDA task, the model needs to generalize across different domains with varying distributions; thus the encoder needs to extract appropriate features that are transferable across domains. The challenges of class-level adaptation are two folds: learning transferable features, and modeling P G T (Z|Y ) without label information.
To solve the first problem, we use MI based methods. Following the InfoMax principle, we maximize the mutual information between features from the same class on the target and source mixture distribution. This encourages the features of the source domain to carry more information about the features of the same class in target domain, and thus provides opportunities for transferring classifier across domains.
As for the second challenge, we first revisit the data shift hypothesis. The distribution of labels P (Y ) remains independent of domains; therefore the key is to model the conditional distribution P (Z|Y ) on the target domain. However, modeling P (Z|Y ) is intractable, since labels on the target domain are inaccessible. To tackle this problem, we model a surrogate distribution Q(Z|Y ) instead.
We introduce the goal of maximizing MI in section 4.2, and theoretically explain how MI affects domain adaptation risk. In Section 4.3, we will introduce the model in detail, including the variational estimation of MI, the modeling of the surrogate distribution, and the optimization of the loss function of the model.

Mutual Information Maximization
MI measures the degree to which two variables can predict each other. Inspired by InfoMax principle , we maximize the MI between the features within the same class. It encourages features from different classes to be discriminable from each other.
We maximize MI between features on both source and target domain, regardless of which domain they come from. So we introduce mixture distribution S + T of both domain, which is Note that because P S (y) = P T (y) = P S+T (y), Define the distribution of features from the same class as which means the feature z 1 and z 2 are sampled independently from the conditional distribution of the same class, with equal probability from source domain and target domain.
MI between features is maximized within the mixture distribution, as formalized bellow: However, due to the lack of target domain labels, P G S+T is hard to model and thereby it is infeasible to estimate I G S+T directly. To address this problem, we propose a surrogate joint distribution Q(Z, Y ) as the substitute for target domain P G T . Then the mixture distribution becomes P G S+Q = 1 2 (P G S + Q), and the objective becomes maximizing I G S+Q (Z 1 ; Z 2 ). The construction and optimization of the surrogate joint distribution is explained in Section 4.3.

Theoretical Motivation for MI Maximization
We use theoretical bound to demonstrate the motivation for using MI maximization. Our theoretical results prove that minimizing the expected risk on the target domain can be naturally transformed into MI maximization and expected risk minimization on the source domain, which explains why MI maximization is pivotal to our framework. The proofs are in appendix.
Definition 1 (H∆H-Divergence). Let F 1 ∈ H, F 2 ∈ H be two hypotheses in hypothesis space H : Z → Y.
. H∆H-divergence, which is the discrepancy of two distributions P 1 , P 2 w.r.t. any hypothesis Theorem 1 (Bound of Target Domain expected risk). The expected risk on target domain can be upper-bounded by the negative MI between features, and H∆H -divergence between features of two domains: The proof is in appendix. We give an explanation of the conditions for the upper bound to be equal.
, and it measures how much uncertainty of Y is reduced by knowing the feature, and it's . This upper bound decomposes the cross-domain generalization error into the divergence of feature marginal distribution and MI of features. It emphasizes that in addition to the divergence of the feature marginal distributions, only a MI term is enough for knowledge transfer across domains.
In this work, we minimize the expected risk on the source domain and maximize MI, for minimizing the upper bound of expected risk on target domain. Due to the lack of labels on target domain, we estimate MI based on surrogate distribution Q. The expected risk upper bound based on surrogate MI is further derived as follows.
| where B is the set of measurable subsets under P 1 and P 2 . Theorem 2 (Bound Estimation with Surrogate Distribution).
) be the bias of surrogate distribution Q w.r.t target distribution. The expected risk on target domain can be upper-bounded by the negative surrogate MI between features, H∆H -divergence between source and target domain, and additional bias of surrogate domain: The proof is in appendix. This theorem supports the feasibility of domain adaptation via maximizing surrogate MI , where the first term is the distance between the surrogate and target feature marginal distribution, and the second term is the risk of conditional label surrogate distribution. To minimize the upper bound, the bias of the surrogate distribution should be small. Bias equal to zero if and only if surrogate feature distribution and conditional label distribution are the same as target distribution, i.e., P G T = Q, where surrogate distribution does not introduce errors.

SIDA Framework
We employ MI maximization and surrogate distribution in our SIDA framework, as shown in Figure 2. During training, a surrogate distribution is first built from target and source data via optimizing w.r.t. Laplacian and MI. Then a mixture data distribution is created by encoding source data to features and sampling target features from the surrogate distribution. The encoder is optimized by maximizing MI, and minimizing classification error. The overall loss is: We elaborate each module in the following sections, and introduce the optimization of surrogate distribution in the last sections.

Mutual Information Estimation
Several MI estimation and optimization methods are proposed in deep learning . In this work, we use the following variational lower bound of MI as proposed in : where f is a score function in Z × Z → R. The equality holds when The proof is in appendix. Therefore maximizing MI can be transformed into maximizing its lower bound, and the loss is: where f (z 1 , z 2 ) is constructed as T m2 m1 (|z 1 − z 2 | 2 ). T m2 m1 is a threshold function, i.e., T m2 m1 (a) = max(m 1 , min(m 2 , a)).

Surrogate Distribution Construction
We decompose the surrogate distribution Q(Z, Y ) into two factors Q(Z, Y ) = Q(Y )Q(Z|Y ), and describe the construction of two factors individually.
According to the data shift assumption, P T (Y ) is similar to P S (Y ), thus Q(Y ) should be similar to P S (Y ). However, source distribution may suffer from the class imbalance problem, which will harm the performance on classes with fewer data. A common solution to this problem is class-balanced sampling, which samples data on each class uniformly. In this work, for the balance across different classes, the marginal distribution P S (Y ) and Q(Y ) are both considered as uniform distribution.
As for the second term, the conditional surrogate distribution Q(Z|Y ) is constructed by weighted sampling method. We need to construct the Q(Z|Y ) to calculate Eq. 10, which takes the form of expectation, and only needs samples from Q(Z|Y ) to estimate. Instead of explicitly modeling Q(Y |Z), we use the ideas of importance sampling. For each class, the surrogate conditional distribution Q(Z|y j ) is constructed by weighted sampling from target features. Thus Q(Z|Y ) is a distribution on target features{G(x i T )} n T i=1 , and parameterized by W ∈ R n T ×n Y , where n Y is the number of labels: Compared with pseudo-labeling, our estimation method has the following advantages: (1) The surrogate marginal distribution of feature Q(Z) = Y Q(Z|Y )P (Y ) is not fixed, which enables us to select features more flexibly.
(2)The construction process of the surrogate distribution makes MI estimation I(Z 1 , Z 2 ) more convenient. Our surrogate distribution Q(Z|Y ) provides weights so that weighted sampling can be performed directly.
The challenge is to optimize the sampling probability weights W ij so as to minimize the bias of the surrogate distribution. We propose to optimize this distribution via Laplacian regularization as well as MI, which is explained in details in the following section.

Surrogate Distribution Loss
Inspired by semi-supervised learning, we expect that the surrogate distribution is consistent with the clustering structure of the feature distribution, based on the assumption that the feature is well-structured and clustered according to class, regardless of domains. We employ Laplacian regularization to capture the manifold clustering structure of feature distribution.
Let A ∈ R n T ×n T be the adjacent matrix of target features, where the entry A ij measures how similar G(x i T ) and G(x j T ) are, and D = Diag(A1) is the degree matrix, i.e. D ii = j A ij and D ij = 0, ∀i = j. We construct A as K-nearest graph on target features, and the Laplacian regularization of W is defined as where L is the normalized Laplacian matrix L = I − D − 1 2 AD − 1 2 . This regularization encourages W ik and W jk to be similar if feature G(x i T ) is similar to G(x j T ). It also enables the conditional surrogate distribution to spread uniformly on a connected region.

Classification and Auxiliary Loss
The model is optimized in supervised manner on the source domain. The classification loss is the standard cross-entropy loss via class-balanced sampling.
And we use auxiliary classification loss on pseudo-labels from the surrogate distribution, as the classifier will benefit from label information of the surrogate distribution. We use mean square error (MSE) for pseudo-labels, which is more robust to noise than cross entropy loss.

Optimization of Surrogate Distribution
We optimize both L Laplacian and L MI w.r.t. W for a structured and informative surrogate distribution. At the beginning of each epoch, W is initialized by K-means clustering and filtered by the distance to the clustering centers, i.e. W i,j = 1 µj nearest to G(xi) 1 d(G(xi),µj )<θ , where µ j is the j-th clustering center during clustering, and normalized as To minimize two losses w.r.t W , the gradients are derived analytically. The derivation is in appendix.
Based on the gradient of these two losses, we perform Tstep descent update of W with learning rate η 1 and η 2 respectively, and each step we project W back to the probability simplex. See appendix for details.

Experiments
In this section, We evaluate the proposed method on three public domain adaptation benchmarks, compared with recent state-of-the-art UDA methods. We conduct extensive ablation study to discuss our method.

Datasets
VisDA-2017  is a challenging benchmark for UDA with the domain shift from synthetic data to real imagery. It contains 152,397 training images and 55,388 validation images across 12 classes. Following the training and testing protocol in , the model is trained on labeled training and unlabeled validation set and tested on the validation set.
Office-31 [Saenko et al., 2010] is a commonly used dataset for UDA, where images are collected from three distinct domains: Amazon (A), Webcam (W) and DSLR (D). The dataset consists of 4,110 images belonging to 31 classes, and is imbalanced across domains, with 2,817 images in A domain, 795 images in W domain, and 498 images in D domain. Our method is evaluated on all six transfer tasks. We follow the standard protocol for UDA [Long et al., 2017b] to use all labeled source samples and all unlabeled target samples as the training data.
Office-Home [Venkateswara et al., 2017] is another classical dataset with 15,500 images of 65 categories in office and home settings, consisting of 4 domains including Artistic images (A), Clip Art images (C), Product images (P) and Real-World images (R). Following the common protocol, all 65 categories from the four domains are used for evaluation of UDA, forming 12 transfer tasks.

Implementation details
For each transfer task, mean (±std) over 5 runs of the test accuracy are reported. We use the ImageNet pre-trained ResNet-50  without final classifier layer as the encoder network G for Office-31 and Office-Home, and ResNet-101 for VisDA-2017. The details of experiments are in appendix. The code is available at https://github.com/zhaoht/SIDA.

Baselines
We compare our approach with the state of the arts. Domain alignment methods include DAN [Long et al., 2015], DANN [Ganin et al., 2016], JAN [Long et al., 2017b]. Class-level methods include conditional alignment methods (CDAN , DCAN , ALDA [Chen et al., 2020a]), and contrastive methods (DRMEA , ETD [Li et al., 2020a], DADA , SAFN ). We only report available results in each baseline. We use NA, DA, CA, CT to note no adaptation method, domain alignment methods, conditional alignment methods and contrastive methods respectively.

Results and Comparative Analysis
In this section we will present our results and compare with other methods for evaluation on three standard benchmarks mentioned earlier. We report average classification accuracies with standard deviations. Results of other methods are collected from original papers or the follow-up work. We provide visualizations of the features learned by the model in the appendix. VisDA-2017 Table 1 summarizes our experimental results on the challenging VisDA-2017 dataset. For fair comparison, all methods listed here use ResNet-101 as the backbone network. Note that SIDA outperforms baseline models with an average accuracy of 84.0, surpassing the previous best result reported by +4%.
Office-31 The unsupervised adaptation results on six Office-31 transfer tasks based on ResNet-50 are reported in Table 2. As the data reveals, the average accuracy of SIDA is 90.4, the best among all compared methods. It is noteworthy that our proposed method substantially improves the classification accuracy on hard transfer tasks, e.g. W→A, A→D, and D→A, where source and target data are not similar. Our model also achieves comparable classification performance on easy transfer tasks, e.g. D→W, W→D, and A→W. Our improvements are mainly on hard settings.
Office-Home Results on Office-Home using ResNet-50 backbone are reported in Table 3. It can be observed that SIDA exceeds all compared methods on most transfer tasks with an average accuracy of 71.2. The performance reveals the importance of maximizing MI between feature in difficult domain-adaptation tasks which contain more categories.
In summary, our surrogate MI maximization approach achieves competitive performance compared to traditional alignment based methods and recent pseudo-label based methods for UDA. It underlines the validity of using information theory methods for UDA via MI maximization.

Ablation Study
In this section, to evaluate how different components of our work contribute to the final performance, we conduct ablation study for SIDA on Office-31. We mainly focus on harder transfer tasks, e.g. A→W , A→D, D→A and W→A. We investigate different combinations of two components:MI maximization and surrogate distribution (SD). Note that without surrogate distribution, we use pseudo label computed by the same method as surrogate distribution initialization to estimate MI. The average classification accuracy on four tasks are in Table 4. From the results, we can observe that the model with MI maximization outperforms the base model without the two components by about 2.5% on average, which demonstrates the effectiveness of the maximization strategy. The surrogate distribution also improves the average performance by 1.1% compared to base model, confirming that the surrogate distribution improves the estimation quality of target domain compared to pseudo label method. The combination of two components yields the highest improvement.

Conclusion and Future Work
In this work, we introduce a novel framework of unsupervised domain adaptation and provide theoretical analysis to validate our optimization objectives. Experiments show that our approach gives competitive results compared to state-ofthe-art unsupervised adaptation methods on standard domain adaptation tasks. One unresolved problem is to integrate the domain discrepancy in target risk upper bound into mutual information framework. This problem is left for future work. Theorem 3 (Bound of Target Domain expected risk). The expected risk on target domain can be upper-bounded by the negative MI between features, and H∆H -divergence between features of two domains: Proof. The risk can be relaxed by triangle inequality For the term P G T (F ) + P G S (F ), we have While F can be any classifier, define F as the optimal classifier on P G S+T , i.e. F (z) = arg max y P G S+T (y|z). Recall the definition of MI, Which means that According to the MI chain rule, I(Z 1 ; Y, Z 2 ) = I(Z 1 ; Y ) + I(Z 1 ; Z 2 |Y ) = I(Z 1 ; Z 2 ) + I(Z 1 ; Y |Z 2 ). Since Z 1 and Z 2 are two samples from class Y , Z 1 and Z 2 are independent for a given Y , i.e., I(Z 1 ; Z 2 |Y ) = 0. So we can get I(Z 1 ; Y ) = I(Z 1 ; Z 2 ) + I(Z 1 ; Y |Z 2 ). Because I(Z 1 ; Y |Z 2 ) ≥ 0, we finally get Note that I(Z 1 ; Y ) is I(Z; Y ), because Z 1 and Z 2 both follow distribution P (Z|Y ).
So now we get the conclusion by We give an explanation of the conditions for the upper bound to be equal. I G S+T (Z 1 ; Z 2 ) is a lower bound of I G S+T (Z; Y ), and it measures how much uncertainty of Y is reduced by knowing the feature, and it's equal to H(Y ) if and only if P . Thus if the H∆H -divergence is zero, i.e., P G S (Z) = P G T (Z), then it's ensured that P G S (Z, Y ) = P G T (Z, Y ), and P G T (F ) = P G S (F ).

Proof for Theorem 4.2
Theorem 4 (Bound Estimation with Surrogate Distribution). Let B = d 1 (P G T (Z), Q(Z)) + P G T (Q(Y |Z)) be the bias of surrogate distribution Q w.r.t target distribution. The expected risk on target domain can be upper-bounded by the negative surrogate MI between features, H∆H -divergence between source and target domain, and additional bias of surrogate domain: Proof. The expected expected risk can be relaxed by triangle inequality: By the same method as the previous proof, terms Q (F ) + P G S (F ) can be deduced into MI −4I G S+Q (Z 1 ; Z 2 ) + 4H(Y ).

Proof for the Equality Condition of MI Estimation
Proposition 5. The following MI lower bound holds where f is arbitrary function in Z×Z → R. The equality holds when e f (z 1 ,z 2 ) E P (z 1 ) e f (z 1 ,z 2 ) = P (z1|z2) P (z1) and E P (z1) E P (z2) e f (z1,z2) = e.

Details for Surrogate Distribution Optimization
Let P be the conditional distribution matrix of source domain, i,e P ij = P S (x i S |y j S ), and W be the conditional distribution matrix of surrogate distribution Q. Let M = 1 2 P W be the conditional distribution matrix of mixture distribution P S+T .
Let S is the score function matrix, i.e. S i,j = f (G(x i ), G(x j )), i, j = 1, . . . , n S + n T , where f is the score function of the MI lower bound. With class-balanced sampling, L M I can be represented as follows: The gradient w.r.t M is And thus the gradient w.r.t W is Where (0, I) ∈ R n T ,n S +n T ,I is identity matrix with size n T . In practice, we find it harmful to minimize L M I by ∇ W L M I directly, because it will encourage the distribution to concentrate on only a few samples rapidly. We thus adjust the decent direction of L M I to update the distribution slowly. Let |∇ W L Laplacian | be the entry-wise absolute value of ∇ W L Laplacian . The descent directions are: d 1 = −∇ W L Laplacian and d 2 = −|∇ W L Laplacian | (∇ W L M I ), where is entry-wise multiplication. Due the property of ∇ W L Laplacian which is high on the margin of each conditional distribution, this yield a diffusion-like update of W , which prevent rapid collapse of surrogate distribution. Therefore, the update rule of W is W k+1 = Π(W k + η 1 d k 1 + η 2 d k 2 )), where Π is the projection operator onto probability simplex for each column of W. η 1 and η 2 are learning rate. The iteration is performed T times.

Implementation details
For each transfer task, mean (±std) over 5 runs of the test accuracy are reported. We use the ImageNet  pre-trained ResNet-50  without final classifier layer as the encoder network G for Office-31 and Office-Home, and ResNet-101 for VisDA-2017. Following [kan, ], the final classifier layer of ResNet is replaced with the task-specific fullyconnected layer to parameterize the classifier F , and domain-specific batch normalization parameters are used. Code is attached in supplementary materials.
The model is trained in the finetune protocol, where the learning rate of the classifier layer is 10 times that of the encoder, by mini-batch stochastic gradient descent (SGD) algorithm with momentum of 0.9 and weight decay of 5e − 4. The learning rate schedule follows [Long et al., 2017b;Long et al., 2015;, where the learning rate η p is adjusted following η p = η0 (1+ap) b , where p is the normalized training progress from 0 to 1. η 0 is the initial learning rate, i.e. 0.001 for the encoder layers and 0.01 for the classifier layer. For Office-31 and Office-Home, a = 10 and b = 0.75, while for VisDA-2017, a = 10 and b = 2.25. The coefficients of L MI and L Auxilary are α 1 = 0.3, α 2 = 0.1 for Office-31, α 1 = 1.3, α 2 = 1.0 for Office-Home, and α 1 = 3.0, α 2 = 1.0 for VisDA-2017. The hyperparameters of surrogate distribution optimization include K-nearest Graph K = 3, number of iterations T = 3, learning rate η 1 = 0.5 and η 2 = 0.05.
Experiments are conducted with Python3 and Pytorch. The model is trained on single NVIDIA GeForce RTX 2080 Ti graphic card. For Office-31, each epoch takes 80 seconds and 10 seconds to perform inference. Code is attached in supplementary materials.

Visualization
Our visualization experiment is carried out on D → A and W → A tasks in the data set Office31, which are the two most difficult tasks in Office31. The baselines we chose were ResNet-50 pre-trained on ImageNet and CDAN . We chose CDAN because it is a typical conditional domain alignment method. Pre-trained Resnet-50 is fine-tuned on the source domain and then tested on the target domain. Results of CDAN are obtained by running the official code. We train all the models until convergence, then encode the data of source domain and target domain with the model, and take representation before the final linear classification layer as feature vectors. We use t-SNE to visualize the features, using the t-SNE function of scikit-learn with default parameters. The results are in the link. Figure 3 shows the results of task D → A. From top to bottom are the feature visualizations on task D → A of ResNet-50, CDAN, and SIDA, respectively. The left column is the feature comparison of the source and target domains. Red represents the source domain, and blue represents the target domain. The results show that SIDA emphasises discriminability of features. The right column shows the feature of different classes on the target domain. SIDA makes target features better distinguishable. Figure 4 shows the results of task W → A. The results on task W → A are similar to task D → A.
The visualization results show that SIDA can make the features of different categories more distinguishable, a natural consequence of maximizing MI among features from the same category. Thus features can be easier for classification, as the visualization shows.