{'fraction_non_alphanumeric': 0.03645612547689699, 'fraction_numerical': 0.009564434082238237, 'mean_word_length': 4.635264257987459, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Pareto Domain Adaptation

Domain adaptation (DA) attempts to transfer the knowledge from a labeled source domain to an unlabeled target domain that follows different distribution from the source. To achieve this, DA methods include a source classification objective to extract the source knowledge and a domain alignment objective to diminish the domain shift, ensuring knowledge transfer. Typically, former DA methods adopt some weight hyper-parameters to linearly combine the training objectives to form an overall objective. However, the gradient directions of these objectives may conflict with each other due to domain shift. Under such circumstances, the linear optimization scheme might decrease the overall objective value at the expense of damaging one of the training objectives, leading to restricted solutions. In this paper, we rethink the optimization scheme for DA from a gradient-based perspective. We propose a Pareto Domain Adaptation (ParetoDA) approach to control the overall optimization direction, aiming to cooperatively optimize all training objectives. Specifically, to reach a desirable solution on the target domain, we design a surrogate loss mimicking target classification. To improve target-prediction accuracy to support the mimicking, we propose a target-prediction refining mechanism which exploits domain labels via Bayes' theorem. On the other hand, since prior knowledge of weighting schemes for objectives is often unavailable to guide optimization to approach the optimal solution on the target domain, we propose a dynamic preference mechanism to dynamically guide our cooperative optimization by the gradient of the surrogate loss on a held-out unlabeled target dataset. Extensive experiments on image classification and semantic segmentation benchmarks demonstrate the effectiveness of ParetoDA


Introduction
Domain adaptation (DA) is a well-established paradigm for learning a model on an unlabeled target domain with the assistance of a labeled related source domain, which has attracted a surge of interests in the machine learning community [38,37,16]. By bridging the domain gap on the premise of ensuring the performance of source classification task, DA can adapt the model learned from the source domain to the target in the presence of data bias, which solves the dilemma of label scarcity in many real-world applications [59,51,14,49,3,61]. Extensive DA approaches have been proposed in recent years, achieving great performances in many areas such as image classification [14,31,60], semantic segmentation [53,56,19], and object detection [43,47,58].
Most DA methods are devoted to aligning the distributions across domains by explicitly minimizing some discrepancy metrics [30,32,60] or adopting adversarial learning [14,31,45]. However, since the domain alignment objective also depends on the target domain that deviates from the source, the gradient direction of it may conflict with that of classification objective on the source domain. Thus, when the optimization proceeds to a certain stage, one objective will deteriorate if further improving the other one. Under this circumstance, no solution can reach the optimal of each objective at the same time. We can only obtain a set of so-called Pareto optimal solutions instead, where we cannot further decrease all objectives simultaneously [12]. All Pareto optimal solutions in loss space compose Pareto front as shown in Fig. 1.  [14] and CDAN [31] on task W→A (Office-31).
This dilemma obviously has been overlooked by most former DA methods, which generally adopt some empirical weight hyperparameters to linearly combine the objectives, constructing an overall objective. This linear weighting scheme has two major concerns. First, as pointed out in [4], it can only obtain solutions on the convex part of the Pareto front of the objectives, while cannot reach the non-convex part, as shown in Fig. 1 (a)-(b). That is because it only considers reducing the overall objective value, which might damage some objectives during optimization (see the red up-arrow). Unfortunately, the Pareto fronts of DA methods are often non-convex, due to loss conflict caused by domain shift. See examples shown in Fig. 2. Second, one could hardly reach the desired solution that performs best on target domain by tuning the weight hyper-parameters, since the obtained solution will deviate severely when the trade-offs slightly change (see the slopes of the dash lines in Fig. 1 (a)-(b) for reference).
To remedy these issues, we rethink the optimization scheme for DA from a gradient-based perspective. By weighting objectives to control the overall optimization direction so that no objective deteriorates, one can reach a Pareto optimal solution on the Pareto front [11,27,34,36]. Moreover, [34] precisely sets objective weights according to a prefixed preference vector (see Fig. 1 (c)). However, neither of them suits DA, because 1) in DA, the goal is to minimize the target classification loss L * T , but neither L S or L D directly corresponds to this goal; 2) the target domain has no label to construct a classification loss for the goal; 3) prior knowledge of weighting schemes for objectives is often unavailable to guide optimization to approach the optimal solution that minimizes L * T . In this paper, we propose a Pareto Domain Adaptation (ParetoDA) approach, which cooperatively optimizes training objectives and is specifically designed for DA. Specifically, to minimize L * T , we design a target-classification-mimicking (TCM) loss L T by the mutual information [20] leveraging the target predictions. In this loss, target predictions act as (soft) labels for themselves. Therefore, the accuracy of the predictions is important to support the mimicking. To refine the predictions by maximally exploiting the information at hand, we propose a target prediction refining mechanism which models the domain label as conditional information into the prediction probability by Bayes' theorem. Intuitively, we ensembles a class-wise domain discriminator (trained with the domain labels) as another classifier-only when the class condition is correct, the discriminator can correctly predict the domain. On the other hand, to weight L T , L S and L D to guide the cooperative optimization towards minimizing L * T , we propose a dynamic preference mechanism to model the gradient of L T on a held-out unlabeled target dataset as the guidance. We evaluate the performance on the held-out data because 1) it suggests generalization performance, and 2) it is more stable and convincible than training data. One may be concerned that involving the gradient of the held-out data in training may eventually cause over-fitting of the data. Fortunately, our theoretical analyses show that the held-out data can guide but will not be over-fitted by the optimization of our method. Our contributions are: • We rethink the optimization scheme in existing DA methods, which can only reach restricted solutions if the optimization objectives conflict with each other. And existing gradient-based strategies do not suit DA.
• We propose a Pareto Domain Adaptation method to obtain the desired Pareto optimal solution learned and dynamically guided (using the held-out unlabeled target data) by the designed surrogate objective function that mimics target classification. To better support the mimicking, we also propose a target prediction refining mechanism. As a general technique, ParetoDA can be plugged into various DA methods and enhance their performance.
• We evaluate ParetoDA on two DA applications: image classification and semantic segmentation. Experimental results demonstrate the effectiveness of ParetoDA.

Background
Consider m objectives, each with a non-negative loss function L i (θ) : where θ is the shared parameters of the objectives to be optimized. There may exist no single solution that can reach the optimal solution of each objective at the same time due to they may conflict with each other. What we can obtain instead is a set of so-called Pareto optimal solutions, the related definitions are as follows [62]: . We say that θ 1 dominates θ 2 iff θ 1 ≺ θ 2 . Note that θ 1 ⊀ θ 2 if θ 2 is not dominated by θ 1 .
Pareto optimality. If a solution θ 1 dominates θ 2 , then θ 1 is clearly preferable as it perform better or equally on each objective. A solution θ * is Pareto optimal if it is not dominated by any other solutions. That is, θ ⊀ θ * , ∀θ ∈ R d − {θ * }.
Pareto front. The set of all Pareto optimal solutions in loss space is Pareto front, where each point represents a unique solution.
Preference vector. A Pareto optimal solution can be viewed as an intersection of the Pareto front with a specific direction in loss space (a ray). We refer to this direction as the preference vector.
In gradient-based optimization, we can reach a Pareto optimal solution by starting from an arbitrary initialization and iteratively finding the next solution θ t+1 that dominates the previous one θ t , i.e., θ t+1 = θ t − ηd des , where η is the step size and d des is the optimization direction. Let g i = ∇ θ L i be the gradient of the i-th objective function at θ, the descent direction d des is given by . In other words, the descent direction d des should optimize or not damage each objective. Thus, moving against d des , starting from θ, amounts to a decrease in the objective values. When d T des g i = 0, the optimization process ends, implying that no direction can further optimize the all the objectives simultaneously. At this moment, if we still want to improve the performance of a specific objective, other objectives will deteriorate. This paper focuses on dynamically searching for a desired Pareto optimal solution with the aid of the proposed TCM loss in DA problems, aiming to enhance the performance on the target domain.

Related Work
Domain Adaptation aims to transfer knowledge across domains. The theory proposed by Ben-David et al. [1] reveals that the key ideology of DA is to diminish the domain gap between two domains, enabling successful transfer. Existing DA methods mainly mitigate domain shift via minimizing some statistic metrics [30,60,32,48] or adversarial training [14,31,45,24,26]. To name a few, DAN [30] mitigates the layer-wise discrepancy via Maximum Mean Discrepancy (MMD) metric. MDD [60] introduces Margin Disparity Discrepancy with rigorous generalization bounds to measure the discrepancy between domains. On the other hand, inspired by GANs [17], DANN [14] introduces a domain discriminator to enforce the features from two domains to be indistinguishable. CDAN [31] further extends this idea by integrate classification information into domain discriminator, leading to precise domain alignment. By contrast, we adopt domain discriminators to refine the target predictions with domain labels. In addition, these methods directly optimize the linear combination of all objectives, while overlooking on the potential conflict between them. In such case, the linear scheme is hard to tune and may only access to restricted solutions on the convex part of pareto front [4]. We propose a gradient-based optimization scheme to search the desirable Pareto optimal solution on the entire pareto front without weight hyper-parameters, which is orthogonal to prior DA methods.
Weight-Based Optimization is an intuitive strategy to optimize multiple objectives at the same time, which unifies all the training objectives using weight hyper-parameters to form an overall objective. Since it is hard to manually assign proper weights in prior, many DA methods conduct Deep Embedded Validation [57] to select weight hyper-parameters. Recently, some methods propose to dynamically weight the objectives leveraging gradient magnitude [7], the rate of change in losses [29], task uncertainty [22], or learning non-linear loss combinations by implicit differentiation [35]. However, these weight-based methods focus on seeking a balanced solution, which is not bound to be optimal for target classification in DA. By contrast, we explore the gradient-based optimization, aiming to find an optimal solution for DA problems.
Multi-Objective Gradient-Based Optimization leverages the gradients of objectives to reach the final Pareto optimal solution. MGDA [11] proposes to simply update the shared network parameters along a descent direction which leads to solutions that dominate the previous one. Nonetheless, MGDA may reach any Pareto optimal solution since it has no extra control for the optimization direction. To ameliorate this problem, PMTL [27] splits the loss space into separated sub-regions based on several preference vectors, and achieve Pareto optimal solutions in each sub-region of the Pareto front. Further, EPO [34] can reach a exact Pareto optimal solution by enforcing the decent direction correspond a specific preference vector. However, prior knowledge of which Pareto solution is the optimal one for target classification is often unavailable in DA. In this paper, we propose ParetoDA algorithm to dynamically searching the desirable Pareto optimal solution for target domain guided by a introduced TCM loss on a held-out target data.
.., K} is the corresponding label of the source data x s i . Both source and target domains share one identical label space Y s that contains K classes.
Existing DA methods are mainly devoted to establishing domain alignment by explicitly minimizing the domain discrepancy or adversarial training, which have obtained promising results. Discrepancybased methods attempt to find a metric that can precisely depict the domain discrepancy, then adopt it to minimize the domain gap. While adversarial-based methods play a mini-max game between a feature extractor and a discriminator, where the feature extractor tries to fool the discriminator, leading to domain-invariant features. All in all, these approaches can be decomposed as follows: Discrepancy-based: min Adversarial-based: min where L S is the source classification loss, L D is the domain alignment loss, and θ, φ c , φ d are the parameters of shared feature extractor, classifier and domain discriminator, respectively.
However, the optimization of L D may conflict with that of L S due to the domain shift, under which circumstances the de facto optimization scheme can only access restricted Pareto optimal solutions. Moreover, our goal is to minimize the target classification loss L * T , but neither L S or L D directly corresponds to this goal. Therefore, to search for a desirable Pareto optimal solution for this goal, we propose a gradient-based optimization method for DA. Concretely, we first design a novel surrogate loss that mimics target classification loss, and then leverage the gradient of it on a held-out target dataset to guide the cooperative optimization of objectives.

Target-Prediction Refining Mechanism
The goal of ParetoDA is to cooperatively optimize the multiple objectives in DA, aiming to minimize the target classification loss L * T , which is inaccessible due to the absence of target labels. To tackle this issue, we design a TCM loss L T by the mutual information [20] leveraging the target predictions. In this loss, target predictions act as (soft) labels for themselves. Therefore, the accuracy of the predictions is important to support the mimicking.
Former methods generally adopt the outputs of classifier as target predictions, without considering that the domain labels might be beneficial for classification. To refine the predictions by maximally exploiting the information at hand, we propose to model the domain label as conditional information into the prediction probability by Bayes' theorem. Formally, the class-conditional probability where the class k ∈ {1, . . . , K} and the domain label d ∈ {0, 1}, K is the number of classes, 0 and 1 denote the source and target domain respectively. Based on Bayes' theorem, we have: Here, we adopt a multi-layer perception (MLP) network to model where v k denotes the parameters of the k-th domain discriminator corresponding to class k, which aims to distinguish the domain label of the samples from class k. ρ k|d can be regarded as a multiplicative ensemble of two predictions: p(z=d|x,v k ) k p(z=d|x,v k ) and p(y=k|x,θ,φc) k p(y=k |x,θ,φc) . We interpret how the former prediction works for target classification: for one specific class-wise discriminator, it can distinguish the target domain from the source well only when the input samples contain the corresponding class information, i.e., belong to the corresponding category. Otherwise, the probability of predicting samples as the target will be low.
We train all the class-wise discriminators with (derivations are in the supplementary material): Note that the s k|d here only acts as a weighting scalar, and for source domain s k|d = I(y = k) is hard class label (ground truth label) while for target domain, s k|d = ρ k|d is soft class label (refined prediction). This objective is from an Expectation-Maximization derivation.
Now we acquire refined predictions ρ k|d for target samples which can substitute for the original predictions. As discussed in [15,46], the ideal target predictions should be individually certain and globally diverse, which can be obtained by maximizing the mutual information between input samples and the predictions. Thus, we adopt the information maximization (IM) loss [20] on our improved version prediction ρ k|d , the formulation is as follows: where ρ k|1 denotes p(y = k | z = 1, x, θ), i.e., the probability that the target sample x is classified into class k under the prior condition that it is divided into target domain, andρ k|1 denotes E x∈Dt ρ k|1 , represents the mean of the k-th elements in the outputs of the whole target domain.
Hence, we take L T as the surrogate to mimic the target classification. This loss can be directly added to original DA methods to assist the model adaption to the target domain. More importantly, we leverage it to guide the search for the desirable Pareto optimal solution.

Dynamic Preference Mechanism
This section introduces how we establish gradient-based optimization for DA to handle the inconsistency among objectives and how we dynamically guide the optimization direction towards the desired Pareto optimal solution.
Given losses L S , L D and L T , in each optimization step, we first model the update direction d as a convex combination of gradients of the three losses, i.e., where θ are the shared parameters, i.e., the parameters of the shared feature extractor. A fundamental goal of gradient-based optimization is to find a d to simultaneously minimize all the losses. That is, the network can be optimized along a consistent direction and learn the shared knowledge between source and target domains efficiently and stably. Therefore, the direction d first need to satisfy the constraint Recently, a state-of-art method EPO [34] further controls the direction to reach a Pareto optimal solution with designated trade-off among objectives, using a preference vector fixed apriori. Concretely, EPO guides the optimization direction via minimizing the KL divergence between a normalized loss vector weighted by the preference vector and a uniform probability vector. Since the prior knowledge for the trade-off among objectives in DA is often unavailable and may vary with methods and training procedures, to weight L T , L S and L D to guide the cooperative optimization towards minimizing L * T , we propose to model the gradient of L T on a held-out unlabeled target dataset as the optimization guidance. We evaluate the performance on the held-out data because 1) it suggests generalization performance, and 2) it is more stable and convincible than training data. In practice, we randomly split 10% data from the original target set as the validation set and the rest 90% are taken as the training set. Note that we do not use any ground truth labels on the validation set, instead we calculate the proposed TCM loss on it, and then leverage this validation loss to guide the optimization direction.
Formally, we denote by L V al the TCM loss on the held-out data, then we can directly obtain the gradient descent direction:ĝ v = ∇ θ L V al . Thus we can replace the guidance direction d bal 1 in EPO byĝ v as a dynamical guidance of the optimization direction. For clarity, the optimization problem can be formulated as a linear programming (LP) problem: is an indicator function, 1 ∈ R m is a vector whose elements are all 1, One may be concerned that involving the gradient of the held-out data in training may eventually cause over-fitting of the data. Fortunately, we use the following result to show that the held-out data can guide but will not be over-fitted by the optimization of our method. Theorem 1. Let w * be the solution of the problem in Eq. (6), and d * = Gw * be the resulted update direction. If L V al = 0, then the dominating direction d * becomes a descent direction, i.e., On the other hand, if L V alid > 0, Let γ * = (d * ) Tĝ v be the objective value of the problem in Eq. (6).
The proof of Theorem 1 is deferred to the supplementary material. Note that we add a small = 1e−3 to relax the condition L V al > 0 and L V al = 0 in Eq. (6) as L V al > and L V al ≤ in practice, since the guiding significance of L V al becomes trivial when it is less than a small value.
Algorithm 1 Update Equations for ParetoDA 1: Input: Shared parameters θ t ∈ R d , step size η 2: Calculate objective values: L S , L D , L T 3: Calculate guidance loss on target validation set: L val (using Eq. (5)) 4: Calculate the gradients of training objectives: Calculate the gradient of the guidance loss: g v = ∇ θ t L val 6: Obtain w * by solving LP (Eq. (6)) 7: Optimization direction: d * = Gw * 8: Output: Updated parameters θ t+1 = θ t − ηd * Now we interpret the learning mechanism in Eq. (6). When L V al = 0, the learning is in the pure descent mode, in which the update direction d * = Gw * approximates the mean gradient G1/m, and d * can decrease all the training losses simultaneously. Whereas when L V al > 0, the learning is in the guidance descent mode, in which d * approximatesĝ v . Since d * = Gw * is a convex combination of columns in G, then γ * > 0 means thatĝ v is consistent with some columns in G, and d * is forced to decrease the loss whose gradient is the most consistent withĝ v -which means the highest contribution to the model performance. On the other hand, γ * ≤ 0 means none of the columns in G is consistent withĝ v . In this case, d * is not forced to consistent withĝ v , but only required to decrease the training losses, while approximatingĝ v to the maximum feasible extent.
In summary,ĝ v can dynamically guide the optimization direction towards the desired Pareto solution, but will not over-fit the held-out data, and we no longer need prior knowledge to set a preference vector. The theoretical guarantee for the existence of a Pareto optimal is deferred to the supplementary material. And the updating algorithm of every training iteration is presented in Alg. 1. Note that this optimize scheme is only applied to the shared parameters.
Time Complexity. Let m ∈ Z + be the number of objective, and d ∈ Z + the dimension of model parameters. The computation of (Gw) Tĝ v of (Gw) T G both has run time O(m 2 d). With the best LP solver [8], our LP (Eq. 6), that has m variables and at most 2m + 1 constraints, has a runtime 2 of O * (m 2.38 ). As in DA, we usually have d m, our total run time is Theoretical Insight. In Domain adaptation literature, the theory proposed by Ben-David et al. [1] provides an upper bound for the generalization error on target data, which is composed of three terms: 1) the expected error on source domain S (h); 2) the H∆H divergence between two domains d H∆H (D s , D t ); 3) the combined error λ of the ideal joint hypothesis h * . The formulation is: where λ = S (h * ) + T (h * ) and h * = arg min h∈H S (h) + T (h).
The goal of DA methods is to lower the upper bound of the expected target error. S (h) is expected to be small thanks to the supervision in source domain. And λ is also generally considered sufficiently small [45,31]. Thus, most DA methods mainly focus on reducing d H∆H (D s , D t ) by domain alignment. However, if one of the three terms goes down and another term goes up, the model still can not effectively reduce the upper bound of T (h). Excessive either source classification or domain alignment will cause damage to the other.
In our ParetoDA, by adopting gradient-based optimization, we can simultaneously optimize the source classification loss and the domain alignment loss, so as to make S (h) and d H∆H (D s , D t ) decrease synchronously. Moreover, both the training of our proposed TCM loss on target data and the optimization guidance of this loss on held-out data contribute to the classification ability of the model to the target domain, and then reduce the combined error λ of the ideal joint hypothesis h * . Consequently, the upper bound of T (h) can be effectively reduced in our work, which validates the effectiveness of ParetoDA theoretically.

Dataset and Setup
We evaluate ParetoDA on three object classification datasets and two semantic segmentation datasets. Office-31 [44] is a typical benchmark for cross-domain object classification involving three distinct  [40], and we adopt the common-used stochastic gradient descent (SGD) optimizer with momentum 0.9 and weight decay 1e-4 for all experiments. Following the standard protocols in [31,14,33], we take the average classification accuracy and mean IoU (mIoU) for image classification and semantic segmentation as evaluation metrics, respectively. More details are presented in the supplementary material due to limited space. Office-Home. Compared with Office-31, Office-Home is a more difficult dataset, whose results are shown in Table 2. In this difficult scenario, ParetoDA can still enhance the performance of former DA methods, with 11.8% and 4.8% improvement on DANN and CDAN respectively. Based on these promising results, we can infer that ParetoDA can reach a desirable Pareto optimal solution which is beneficial for the performance of models.     Table 3. As a plug-in module, BSP improves the basic models by enhancing the discriminability of feature representations. ParetoDA aims to improve the optimization for DA methods, which is also orthogonal to former DA methods. In addition, ParetoDA achieves greater performance gains, further indicating the effectiveness and versatility of our method.

Experimental Results
GTA5 → Cityscapes. Compared with classification tasks, segmentation demands more cost for the pixel-level annotation. Hence, it's of vital importance to conduct domain adaptation for semantic segmentation. We evaluate our method on the adaptive semantic segmentation task, GTA5 → Cityscapes, aiming to transfer the knowledge from synthesized frames to real scenes. The segmentation results are exhibited in Table 4. We apply ParetoDA on two methods, i.e., AdvEnt and FDA, which boosts them by large margins on average. This manifests that ParetoDA can still reach an appropriate optimization solution for enhancing model performance on this pixel-level transfer scenario. Besides, ParetoDA is not limited to handle the conflict between the classical source classification loss and domain alignment loss, our framework can also be applied to where the explicit domain alignment module doesn't exist or there exist multiple losses. For instance, we adopt ParetoDA on a self-training based domain adaptation method CBST [63] and it is clear that ParetoDA is still effective.  Table 5. TCM performs better than IM, demonstrating that integrating domain labels is beneficial for target predictions. Applying the conventional Pareto optimization method [34] with fixed preference vector on DANN achieves considerable performance gain. This reveals that the gradient-based optimization method is superior to the linear weighting scheme. Further, with our proposed ParetoDA, DANN obtains the best results, validating the importance of dynamical preference learning and that TCM loss serves as a better guidance for optimization. Optimization Path Visualization. To intuitively verify that ParetoDA can dynamically search the desired Pareto optimal solution under the guidance of our proposed loss, we visualize the optimization paths of different optimization schemes (see Fig. 3(a)). Note that all these schemes start from the same initial point, but we only plot several solutions of the final stage of optimization for better illustration. It is clear that the linear weighting scheme not only deteriorates one of the objectives at some steps, but also reaches a solution on the convex part of the Pareto front, which deviates severely from the ideal one. The conventional gradient-based optimization method EPO (with fixed preference vector r = [1, 1, 1]) controls the optimization direction to avoid damaging any of the objectives. However, its solution heavily relies on the pre-defined preference vector for training objectives, which is ambiguous in DA. By contrast, our proposed ParetoDA further dynamically adjusts the optimization direction with the gradient direction of the proposed TCM loss. For comparisons, we illustrate the ideal solution guided by the supervised classification loss on target domain. One can observe that the optimization direction of ParetoDA is consistent with the ideal one in general, and reach a final solution closest to the ideal one.

Analytical Experiments
Sensitivity to Different Objective Scales. The scale of the objectives may vary with methods or datasets. To test the adaptivity of ParetoDA to different objective scales, we conduct experiments on a random task W→A (Office-31) based on DANN by varying λ 0 , λ 1 ∈ {0.1, 0.5, 1.0, 1.5}, which represent the scale factor of object classification objective and domain alignment objective respectively. Fig. 3(b) shows that ParetoDA is not sensitive to the scale variety of objectives, which indicating that ParetoDA is robust across different methods and datasets.
Adaptivity to Deeper Backbone Networks. In this work, we reach a desirable Pareto optimal solution for the target domain by controlling the optimization direction. Whereas on deeper networks, it might be more difficult to control the optimization direction due to the increased complexity.

Conclusion
In this paper, we rethink the optimization scheme for DA from a gradient-based optimization perspective. We present a Pareto Domain Adaptation (ParetoDA) to dynamically search a desirable Pareto optimal solution for boosting model performance, which is orthogonal to former DA methods. Extensive experiments on both classification and segmentation tasks validate the effectiveness of ParetoDA. It is worth noting that we mainly concentrate on the traditional domain adaptation setting for clarity, while the generalization to other variants of DA needs to be further explored.

Supplementary Material A Potential Negative Societal Impacts
Our work focuses on domain adaptation and attempts to properly handle the multiple objectives optimization in it from a gradient-based perspective, which further enhance the performance of adaptive models. This method exerts a positive influence on the society and the community, saves the cost and time of data annotation, boosts the reusability of knowledge across domains, and greatly improves the efficiency. However, this work suffers from some negative consequences, which is worthy of further research and exploration. Specifically, more jobs of classification or target detection for rare or variable conditions may be cancelled. Besides, we should be cautious of the result of the failure of the system, which could render people believe that classification was unbiased. Still, it might be not, which might be misleading, e.g., when using the system in an unlabeled domain.

B Experimental Details
We evaluate our method on cross-domain image classification and semantic segmentation datasets.
For the former, we conduct experiments on Office-31, VisDA-2017 and Office-Home datasets. For the latter, we conduct experiments on GTA5 and Cityscapes. We directly quote the results from original papers if the experimental settings are the same. In addition, we report the average result of three random experiments.

B.1 Experiment on image classification
Network architectures We adopt the ResNet-50 and ResNet-101 [18] model pre-trained on Ima-geNet [23] as the backbone network for classification tasks, and replace the last fully connected layer with a bottleneck layer to speed up the experiments [14,31]. The architecture of classifier is simply a single fully connected layers with random initialization attached to the bottleneck layer. And the domain discriminator is composed of three fully connected layers, each followed by a BatchNormalize layer and a Relu activation function. Admittedly, the choice for the classifier and discriminator is arbitrary and better adaptation performance may be obtained if this part of the architecture is tuned.
Training details We fine-tune from ImageNet pre-trained models and train other layers from the scratch with learning rate 10 times that of the feature generator layers. We adopt Stochastic Gradient Descent optimizer (SGD) [2] with learning rate 0.003, momentum 0.9 and weight decay 1 × 10 −4 . Besides, we adopt the annealing procedure mentioned in [31] to schedule the learning rate. All experiments run on a single NVIDIA RTX 3090 GPU for training.

B.2 Experiment on semantic segmentation
Network architectures We leverage the DeepLab-v2 [5] framework with the pre-trained ResNet-101 model as the base feature extractor for segmentation task. To better capture the scene context, Atrous Spatial Pyramid Pooling (ASPP) is used for classifier and applied on the conv5 feature outputs. We fix the sampling rates as {6, 12, 18, 24} following [50] and modify the stride and dilation rate of the last layers to produce denser feature maps with larger field-of-views.
Training details For evaluation metrics, we report pre-class intersection-over-union (IoU) and mean IoU over all classes. We use the evaluation code released along with the Cityscapes dataset, which calculates the PASCAL VOC intersection-over-union, i.e, IoU = T P T P +F P +F N [13], where T P , F P , F N are the amount of true positive, false positive and false negative pixels, respectively, determined over the whole test set.

C Experimental Results with Error Bars
For the sake of objective, we run all the experiments multiple times with random seed. We report the average results in the main body of paper for elegant, and show the complete results with error bars in the form of mean±std below (Table. 7   Proof. If L V al = 0, then its gradient is also a zero vector:ĝ v = 0. This means J is empty. As a result, I(J = ∅) = 0 and all the constraints of Eq. (6) becomes Then d * becomes a descent direction.
On the other hand, we consider the case in that L V al > 0.
since Gw * is the maximum of the convex combination of the columns in G, it means that there is no gradient g j , j ∈ {1, 2, 3}, for which g T jĝ v > 0. As a result, J is empty and I(J = ∅) = 0. This is similar to the case of L V al = 0. So d * becomes a descent direction.

D.2 Derivations for Eq. (4)
Our derivations follow a standard Expectation-Maximization process. Eq. (3) is actually the E-step to estimate the hidden variable, class label, on the target domain. The detailed derivation is in the following.
And Eq. (4) is a part of the M-step.
For the M-step, we maximize the joint likelihood, Then the log likelihood is where s k|d = I(y = k) is for the source domain, and s k|d = ρ k|d is for the target domain.
We decompose the final log likelihood as where the first term is Eq. (4) and learns domain discrimination for each class with a weighted loss (the weight is s k|d ), and the second term learns classification for each domain. For the source domain, the class labels are hard. Whereas for the target domain, the class labels are soft.

D.3 Existence Guarantee for Pareto optimal solution
Similar as in [27], we provide a theoretical guarantee for the existence of a Pareto critical (also named as local Pareto optimal in [34]) solution of our method, where a solution is called Pareto critical if no other solution in its neighborhood can dominate this solution. Specifically, we show by Theorem 2 below that our method will optimize until the solution is Pareto critical and satisfies L V al = 0.
Theorem 2. Letθ be the final output model parameters of our method. Let w * be the solution of the problem in Eq. (6) based onθ, and d * = Gw * be the resulted update direction. We have: • ifθ is not Pareto critical, then d * = 0 ∈ R d ; • ifθ is Pareto critical, we further consider two cases: -If L V al = 0, then d * = 0 ∈ R d or (d * ) T G = 0 ∈ R 3 . In either case, the update is meaningless and will halt. -If L V al > 0, assuming the gradient of L T is consistent with the gradient of L V al , i.e., g T 3ĝv > 0, then d * = 0 ∈ R d . Note that the assumption that g T 3ĝv > 0 for a Pareto criticalθ is reasonable, because whenθ is Pareto critical, the model is usually well-trained, and g 3 ,ĝ v are gradients of the same loss on i.i.d sampled two datasets from the target domain. Then it is reasonable to assume the gradients are consistent.
The proof of Theorem 2 is as following: Proof.
• Ifθ is Pareto critical, we consider two cases.
In summary, we can obtain a Pareto critical solution with simple iterative gradient-based update rule θ t+1 = θ t + ηd.