{'fraction_non_alphanumeric': 0.04179610279582039, 'fraction_numerical': 0.011719853148828015, 'mean_word_length': 4.767263843648209, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Your Classifier can Secretly Suffice Multi-Source Domain Adaptation

Multi-Source Domain Adaptation (MSDA) deals with the transfer of task knowledge from multiple labeled source domains to an unlabeled target domain, under a domain-shift. Existing methods aim to minimize this domain-shift using auxiliary distribution alignment objectives. In this work, we present a different perspective to MSDA wherein deep models are observed to implicitly align the domains under label supervision. Thus, we aim to utilize implicit alignment without additional training objectives to perform adaptation. To this end, we use pseudo-labeled target samples and enforce a classifier agreement on the pseudo-labels, a process called Self-supervised Implicit Alignment (SImpAl). We find that SImpAl readily works even under category-shift among the source domains. Further, we propose classifier agreement as a cue to determine the training convergence, resulting in a simple training algorithm. We provide a thorough evaluation of our approach on five benchmarks, along with detailed insights into each component of our approach.

an additional challenge in MSDA is to tackle the domain-shift and category-shift [61] between each pair of source-domains (Fig. 1b). To this end, auxiliary losses are enforced encouraging the model to learn domain invariant but class-discriminative representations. Ultimately, an appropriate alignment of all the domains in the latent space [43] improves the generalization on the target domain (Fig. 1b).
In this work, we approach the MSDA problem from a different perspective. Since deep models are known to capture rich transferable representations [29,38,62], we ask, is an auxiliary feature alignment loss really necessary? The motivation stems from the observation that deep models exhibit a strong inductive bias to implicitly align the latent features under supervision. This is demonstrated in Fig. 2. Following the prior approaches [43,61], we train domain-specific classifiers (Fig. 1b) and observe that the domains do not align in the latent space (Fig. 2a), which calls for an explicit feature alignment loss. However, when we enforce a classifier agreement on the class label for each input instance (Fig. 2b), we find that the domains tend to align, without requiring an explicit alignment loss.
This motivates us to further explore implicit alignment of latent features for MSDA. We aim to leverage the labeled data from multiple source domains, and the multi-classifier setup (Fig. 1a) employed in MSDA to perform alignment, without incorporating auxiliary components such as a domain discriminator [61,66]. In contrast to learning domain-specific classifier modules, we enforce an agreement among the classifiers (Fig. 1c) to align the domains in the latent space.
Since the target domain is unlabeled, we resort to the class labels predicted by the model being trained (a.k.a. pseudo-labels [25]). The adaptation step encourages the classifiers to agree upon these pseudolabels which enables alignment of the target features with the source features that have classifier agreement owing to label supervision. Accordingly, we name the approach as Self-supervised Implicit Alignment, abbreviated as SImpAl (pronounced "simple"). We observe that even under category-shift, implicit alignment can be leveraged to align the shared categories, without requiring additional components (e.g. fine-grained alignment [5,22,42], adversarial discriminator [61]) or cumbersome training strategies (e.g. to handle arbitrary category-shifts [23,61,63]). We also find that classifier agreement can be leveraged as a cue to determine adaptation convergence.
To summarize, we demonstrate successful MSDA by leveraging implicit alignment exhibited by deep classifiers, corroborating the potential for designing simple and effective adaptation algorithms. We conduct extensive evaluation of our approach over five benchmark datasets, with two popular CNN backbone models (ResNet-50, ResNet-101 [16]) and derive insights from the empirical analysis.

Related Work
Here, we briefly review the related works and refer the reader to [67] for an extensive survey.
a) Single-Source Domain Adaptation (SSDA). Motivated by the seminal work by Ben-David et al. [2,3], a large number of SSDA methods [6,10,11,12,28,29,32] have been proposed, that aim to We train a model (with ResNet-50 [16] backbone) with full label supervision from all the three domains on Office-31 [47], while keeping the classifier heads unique to each domain. Although we find that class discrimination is achieved, each domain forms separate sub-clusters, and does not align in the latent space. (b) Enforcing classifier agreement.
Instead of learning domain-specific classifiers, we enforce the classifiers to agree upon the labels for all the samples. We observe that all the domains tend to align, without enforcing an explicit alignment objective, even under a domain-shift. We aim to leverage this inductive bias of deep models, to perform adaptation.
learn domain-agnostic but class-discriminative representations. Inspired by the GAN framework [13], a popular strategy is to employ adversarial learning [18,20,51,52,56,57,58] that aims to confuse a domain-discriminator, thereby aligning the latent features of the domains. Saito et al. [50] formulate an adversarial objective employing classifier discrepancy. In contrast, we aim to study a simpler approach which circumvents the training difficulties encountered in adversarial learning paradigms.
Recently, consistency based regularizers [8,21,36,20] were proposed for domain adaptation. In our work, classifier agreement can be interpreted as a form of consistency at the output space which acts both as an implicit regularizer and as a means to perform latent space alignment for adaptation.
b) Multi-Source Domain Adaptation (MSDA). Several methods [15,43,61,68] learn domainspecific classifier modules and obtain a weighted ensemble prediction for the target samples, motivated by the distribution weighted combining rule [17,34,35]. Zhe et al. [68] employ an alignment loss between each source-target pair in domain-specific feature spaces. In addition, Peng et al. [43] align each pair of source domains using kernel based moment matching and also propose a variant based on adversarial learning [50]. Xu et al. employ multiple domain discriminators to achieve latent space alignment. In this work, we aim to explore a simple adaptation scheme that leverages implicit alignment in deep models. As a result, our approach is applicable even under category-shift among the source domains, while most prior methods [15,43,68] consider only a shared category set. c) Self-training methods. Pseudo-labeling [25] is a popular semi-supervised learning approach where "pseudo" class labels are assigned to unlabeled samples, typically using classifier confidence [7,49,61,69,70] or nearest neighbor assignment [22,40,48,65], while the model is retrained using such samples. Confidence thresholding [27,49,61] is commonly applied to minimize the noise in pseudo-labels. This introduces a sensitive threshold hyperparameter, requiring labeled target samples or domain expertise for precise tuning. Works such as Zou et al. [69,70], Li et al. [27] and Chen et al. [8] propose various regularizers to improve pseudo-label predictions. Xu et al. [61] incorporate an adversarial alignment loss to mitigate the performance degradation arising from noisy pseudo-labels. In contrast, we aim to exploit classifier agreement to perform adaptation and improve the reliability of pseudo-labels without incorporating additional hyperparameters.
3 Self-supervised Implicit Alignment (SImpAl) Notations. Let X and Y denote the input and the output spaces. We consider n d labeled source si , y cj si ) ∈ X × Y} and a single unlabeled target domain dataset D t = {x t ∈ X }. Each source domain has a label-set C si , and the target label-set is defined as C = ∪ n d i=1 C si with n c classes. We learn a deep neural network model having a CNN based feature extractor f : R 224×224×3 → R 256 , and n d classifier modules h : R 256 → R n d ×nc . For convenience, we denote the output of the network as a matrix M = h • f (x), where • represents function composition. M is obtained by stacking the logits produced by each classifier (see Fig. 3). Overview. As is conventional in the MSDA methods [15,61], the multi-classifier setup is treated as an ensemble of diverse classifiers, and the class probabilities are obtained through a convex combination of each classifier's prediction. The model is first trained with the categorical crossentropy loss imposed on the combined data from all source domains. After a "warm-start", we introduce pseudo-labeled target samples into the training process. The adaptation is performed by enforcing the classifiers to agree on these pseudo-labels. We now describe the approach in detail.

Warm-start with source domains
To adapt the network to the target domain, we use pseudo-labeled target samples. Thus, we first aim to achieve a reliability in pseudo-labels by training the model on all source domains. We call this as the warm-start process, which is performed as follows.  Fig. 3) and define the class probability vector p as a convex combination of the probabilities assigned by each classifier, where, M [i·] represents the i th row vector of the matrix M (i.e. the logits of the i th classifier), and ) is the softmax function. Treating p as the class probability vector, we minimize the categorical cross entropy loss (l ce ) using the labeled source samples, The last term in Eq. 2 represents an upper bound for the categorical cross-entropy loss of the ensemble, and is obtained by applying the Jensen's inequality for convex functions. We consider the formulation in Eq. 2 to drive the classifiers to agree upon the label y cj si for x cj si . Thus, the training objective is, The objective in Eq. 3 is minimized by mini-batch stochastic optimization. Each mini-batch contains an equal number of samples from each source domain. In practice, each classifier is given a distinct random initialization, and is trained with the same set of training samples at each mini-batch. Intuitively, this process gradually enables a higher degree of similarity among the classifiers (Fig. 1c) through an agreement in the predicted class labels for source samples. Note that, both the feature extractor f and the multi-classifier module h are shared across all source domains. This step provides a warm-start to introduce pseudo-labeled target samples into the training.
b) Determining the convergence of warm-start. The next question we address is, how to tell if a model is trained sufficiently for the target domain? Intuitively, we would like to train the model until there is a saturation in the target (pseudo-label) accuracy. However, with unlabeled target samples measuring the pseudo-label accuracy is out of bounds. Thus, we propose the classifier agreement as a criterion to determine the convergence. The classifier agreement for an instance x is defined as, where M = h • f (x), and I(·) is the indicator function that returns 1 when the condition is true, else returns 0. Intuitively, when each classifier predicts the same class for a given sample x, we say that the classifiers "agree". Thus, a(x, f, h) = 1 when classifiers agree, and a(x, f, h) = 0 otherwise.
As we shall show in Sec. 4.2, the target pseudo-label accuracy is higher whenever the classifiers agree [37,64]. Thus, classifier agreement is used to filter out target samples having a higher degree of noise in pseudo-labels. Further, we estimate the fraction of target samples for which there is an agreement in the class predictions among the classifiers. Thus, we define the target agreement rate as, We hypothesize that the performance on target samples attains a saturation when the agreement rate converges. Thus, we determine the warm-start interval based on the convergence of A(·).

Introducing target data
After the warm-start, we introduce target samples into the training process. The pseudo-labels are obtained from the classifier predictions as in Eq. 1, i.e. y cj t = arg max j ). We consider the following strategy for pseudo-labeling. To begin with, we select only those target samples for which there is a classifier agreement, since the labels are seen to be more accurate for such samples (verified in Sec. 4.2). Thus, we obtain a subset Secondly, inspired by curriculum learning [4,70] we form an easy-to-hard sampling strategy for D t . For this purpose, we obtain the average classifier margin as a weight for each target instance, where j an j correspond to the indices of the highest and the second highest logit. Intuitively, w measures a form of confidence in prediction. Target samples that are farther from the decision boundaries receive a higher w (see Fig. 7c for the geometrical interpretation). We show in Sec. 4.2 that, in general, samples with a higher w are more likely to possess correct pseudo-labels. Thus, target samples are sorted based on w, and are fed to the training pipeline in the decreasing order of w. Finally, the pseudo-labels are updated every n e epochs on D t . With this strategy, we formalize the training objective for adaptation using the target samples as, After introducing the target samples from D t , we train on both source and target samples, in alternate mini-batches, i.e. we minimize the objectives in Eq. 3 and Eq. 7 in alternate mini-batches. Finally, the network is trained until the target agreement rate A shows convergence. This enables a simple and effective adaptation pipeline using implicit alignment. The algorithm is given in Algo. 1.

Experiments
We present the results of our approach on five standard benchmark datasets -Office-Caltech, Image-CLEF, Office-31, Office-Home and the most challenging large-scale benchmark, DomainNet.
a) Prior Arts. We compare against Deep Domain Confusion (DDC) [58], Deep Adaptation Network (DAN) [29], Deep CORAL (D-CORAL) [54], Reverse Gradient (RevGrad) [10], Residual Transfer Network (RTN) [32], Joint Adaptation Network (JAN) [31], Maximum Classifier Discrepancy Algorithm 1 SImpAl -Self-supervised Implicit Alignment Update {f, h} using the objective in Eq. 3 Update {f, h} using the objective in Eq. 7 12: if n e epochs on D t are completed then Periodically update pseudo-labels 13: Perform steps 5-6 to recompute D t c) Implementation Details. We implement our approach in PyTorch [41]. We use the Adam [19] optimizer, with learning rate 10 −5 and weight decay 5 × 10 −4 for stochastic optimization. The losses in Eq. 3 and Eq. 7 are alternatively optimized and the target agreement rate (Eq. 5) is periodically monitored for convergence. We set n e = 15 epochs as the update rate for the target pseudo-labels (line 12 in Algo. 1). The total number of training iterations are decided based on the convergence of the target agreement rate A for each dataset. Following prior MSDA approaches [68,43], we use ResNet-50 (SImpAl 50 ) and ResNet-101 (SImpAl 101 ) [16] as the CNN backbone.

Results
We present the results in Table 1. The results for the prior baselines are reported from [43] and [68]. Due to the limits of space, we present the full comparison table for DomainNet in the Supplementary.

Analysis
a) Implicit alignment of features. In Fig. 4a, we plot the t-SNE [33] embeddings of the features at the pre-classifier space (output of f ) for SImpAl. Further, we calculate the Proxy-A distance [2] defined as dist A = 2(1 − 2 ) where is the generalization error of a domain discriminator. In Fig. 4b, we report the dist A value across each source-target pair for 3 different models -1) warm-start model, trained on the source domains, 2) the model after adaptation using SImpAl, 3) an oracle model employing SImpAl, where the target pseudo-labels are replaced by the ground-truth labels. This shows that adaptation using SImpAl effectively reduces the distribution-shift in the latent space. Further, we also demonstrate implicit alignment under large domain-shifts (such as Quickdraw and Real-world domains on DomainNet), which enables applications such as cross-domain image retrieval on an unlabeled target domain. See Suppl. for further analysis on implicit alignment. b) Extension to category-shift. To present a more practical scenario for MSDA, [61] introduced two category-shift settings -overlap and disjoint, where the source domains contain overlapping label sets (i.e. C si ∩ C s i = φ, but C si ∩ C s i = C si ∪ C s i ) and disjoint label-sets (C si ∩ C s i = φ) respectively. In such scenarios, it is vital to prevent mis-alignment of different classes across the source domains to avoid negative transfer [42]. Furthermore, since prior MSDA approaches learn domain-specific classifiers, they require separate mechanisms to obtain class probabilities for the domain-specific and the shared classes separately [61]. However, our approach remains unmodified under the presence of category-shift; as such, each classifier learns all the target classes, and the computation of the class probabilities (Eq. 1) remains unchanged. Fig. 4c shows that category-shift is a challenging scenario where all methods show performance degradation, however SImpAl is found to exhibit a relatively lower degradation in the target performance. This is supported by the observation that even under category-shift, only the shared classes align as shown in Fig. 5. See Suppl. for further analysis. c) Target Agreement Rate. Fig. 6a shows the trend in the target agreement rate (A(D t , f, h)) and target performance as training proceeds. We make two observations. Firstly, we find that A increases during training, indicating that the target samples migrate into the classifier agreement region in the latent space ({f (x t ) | a(x t , f, h) = 1}). This migration is necessary for a successful adaptation since the source domains inherently fall in the classifier agreement region (due to the nature of the source training for warm-start). Secondly, a correspondence between the convergence of the target agreement rate and the target accuracy is seen, which validates our hypothesis that A can be used In the absence of shared classes (Cs 1 ∩ Cs 2 = φ), class-wise alignment is not observed among the sources, which is essential to avoid negative-transfer. Note, in both cases, the target clusters align with the respective source clusters (since C = Cs 1 ∪ Cs 2 ). This demonstrates implicit alignment under category-shift. See Suppl. for wider trends.
as a cue to determine the training convergence. This result is of interest in Unsupervised Domain Adaptation methods where the requirement of target labels has been the de-facto for model selection.
d) Do the classifiers agree on correct pseudo-labels? We also calculate the classifier agreement (and disagreement) for target samples that are pseudo-labeled correctly. Notably, Fig. 7a demonstrates that the classifiers tend to agree on an increasing number of target samples with correct pseudo-label predictions. This motivates the periodic update of D t (Lines 12-13 in Algo. 1), which captures an increasing number of target samples with correct pseudo-labels, as the adaptation proceeds.
e) How accurate are target pseudo-labels? As described in Sec. 3.2, we use classifier agreement to select target samples (D t ) with a higher pseudo-label accuracy. In Fig. 6b, we plot the accuracy of pseudo-labels separately for target samples having classifier agreement (i.e. a(x t , f, h) = 1) and disagreement (i.e. a(x t , f, h) = 0). Clearly, pseudo-labels are more accurate (more reliable) when the classifiers agree. Further, the accuracy on the target samples with agreement, D t , is higher than the accuracy on all target samples, D t (orange curve in Fig. 6b). Thus, the use of D t with a higher accuracy in pseudo-labels plays a key role in gradually improving the target performance.
f) Using curriculum for target samples. We form a curriculum for the target samples using the average classifier margin w(x t , f, h) as a weight. Fig. 7c shows the geometrical interpretation of w, that measures how far into the agreement region a target sample falls. Thus, w can be seen as a measure of the confidence in the prediction. As studied by prior methods [15,45,53], high confidence predictions are often correct. We show this in Fig. 7b where we plot the precision of iterations a) Estimating convergence using target agreement rate

Pseudo-label accuracy
Target Accuracy   Fig. 1). Intuitively, w is a measure of classifier confidence averaged over each classifier. Target samples that are further into the classifier agreement region exhibit a higher value of w.

Office-Home (➝ Rw) Office-31 (➝A) Office-31 (➝ A)
target pseudo-labels at various confidence percentiles (in descending order of w). The accuracy shows a decreasing trend with w, validating our hypothesis that w yields an easy-to-hard curriculum. Although our framework supports confidence thresholding to further minimize the pseudo-label noise, we do not employ thresholds for the main results (Table 1) as it introduces sensitive hyperparameters. See Supplementary for an empirical analysis with confidence thresholding.

Conclusion
In this paper, we demonstrated Self-supervised Implicit Alignment (SImpAl), that serves as a simple method to perform Multi-Source Domain Adaptation (MSDA). We observed that deep models exhibit the potential to implicitly align features under label supervision, even in the presence of domain-shift. We demonstrated the use of classifier agreement in SImpAl -to obtain pseudo-labeled target samples, to perform latent space alignment and to determine the training convergence. Extensive empirical analysis demonstrates the efficacy of SImpAl for MSDA.
Our work can facilitate the study of simple and effective algorithms for unsupervised domain adaptation. The insights obtained from our study can be used to explain the efficacy of a number of related self-supervised approaches. A potential direction of research is to develop efficient adaptation algorithms that are devoid of sensitive hyperparameters. Exploring SImpAl for scenarios such as Universal Domain Adaptation [63] would also be of future interest.

Broader Impact
This work presents a simple and effective solution for Multi-Source Domain Adaptation, that has a two-fold positive impact. First, the method is aimed at improving the performance of prediction models by mitigating the bias caused by domain-shift between the training dataset and the test data encountered when deployed in a real-world environment. This is of growing interest in the machine learning community. Secondly, the insights presented in this work facilitate the study of efficient methods to perform domain adaptation, motivating the innovation of, for instance, energy-efficient methods to generalize deep models. While the method shows promising results under domain-shift, one should be cautious of the use of the pseudo-labeling procedure in the presence of adversarial samples, where the pseudo-labels may be less reliable and may result in performance degradation.

Results on DomainNet
Owing to the limits of space, we present a summary of results on DomainNet in the paper. The full comparison is presented here in Table 1. The results for the prior arts are reported from [9].

Further Analysis on Implicit Alignment
In this section present further analysis on implicit alignment. First, we show wider trends for implicit alignment under category-shift. Then, we present ablations on the training objective (with and without classifier agreement, single classifier head). Finally, we study thresholding schemes.

Implicit Alignment under Category-shift
We find that SImpAl works well even under category-shift. In Fig. 4c of the paper, we compare the relative drop (%) in accuracy under category-shift which is obtained as, 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada.

Sources-only
Sources + Target Our approach exhibits a relatively lower drop in accuracy. To understand this better, we further investigate the latent space alignment of the domains after adaptation using SImpAl. We show two t-SNE plots, 1) Sources-only, i.e. showing alignment among the source domains, and 2) Sources + Target, i.e. showing the alignment among all the domains, corresponding to an adapted model. Fig. 1 shows the latent space alignment for the task A, D, W→C for the Office-Caltech dataset. For the Overlap scenario, we set the number of shared categories to 2, while the number of source-private categories are 3, 3, 2 for the sources Amazon (A), DSLR (D) and Webcam (W) respectively. Clearly, alignment among the sources is observed only for the two shared categories (annotated in Fig. 1). In contrast, for the Disjoint scenario, the source domains A, D, W contain 3, 3, 4 unique classes respectively. Here, none of the source clusters align as expected (since each source has a distinct set of classes). However, in both scenarios, we find that the target domain aligns with at least one source

Overlap Disjoint
Sources-only Sources + Target domain (multiple sources if categories are shared). We find a similar trend across other datasets as shown in Fig. 2 for the I, P→C task of ImageCLEF (Overlap: 4 shared and 4, 4 source-private classes; Disjoint: 6, 6 source-private classes), and in Fig. 3 for the A, D→W task of Office-31 (Overlap: 11 shared and 10, 10 source-private classes; Disjoint: 16, 15 source-private classes).

A D W
These results support the observation that implicit alignment can be leveraged even under categoryshift. Note that, in methods such as moment matching [9] or adversarial alignment [2], there is usually no enforcement of class-level alignment across the domains. Thus, such methods are prone to negative transfer via conditional mis-alignment [4,8], i.e. alignment of dissimilar classes. However, SImpAl enables class-conditional alignment by virtue of the label supervision, which could explain the relatively lower drop in performance under category-shift.

Is classifier agreement necessary?
We now turn towards the key observation that we present in the paper. By enforcing classifier agreement we are able to perform adaptation, while, when we learn domain-specific classifiers following previous MSDA approaches [13,14], the latent space alignment is not seen. Here, we provide further empirical analysis in support of the observation through ablations on SImpAl 50 .
a) Warm-start with domain-specific classifiers. We modify the loss formulation in Eq. 3 of the paper, to train domain-specific classifiers, as follows: where σ denotes the softmax activation function. Essentially, an instance x cj si pertaining to the source domain D si trains the corresponding classifier head (note that the logarithmic term contains the probability of class c j of the classifier corresponding to s i ). This is in line with the prior methods such as [3,13] where the domain-specific classifiers progressively learn to discriminate among the classes in their respective domains. Thus, there is no explicit enforcement of classifier agreement.
We perform warm-start with the loss formulation in Eq. 2 (name this model, "w/o agreement") and compare against another model learned using Eq. 3 of the paper (name this model, "with agreement"). Both models are trained for the same number of iterations under identical conditions. Finally, we test each model's performance on the target domain at warm-start. The multi-run statistics over 3 random seeds are presented in Table 2. The model "with agreement" shows a consistent improvement in performance in each scenario, which suggests that the model generalizes better to the target domain.  Table 3: Proxy A-distance (↓) measured between each pair of domains for the warm-start models "with agreement", "w/o agreement" and "single classifier". Note that the model "with agreement" consistently exhibits lower distA than "with agreement", suggesting that it aligns the domains to a greater extent (Sec. 2.2a). Furthermore, we perform an ablation by using a single classifier head which also shows lower distA (Sec. 2.2b).  To uncover the underlying effect, we measure the Proxy where is the generalization error of a domain discriminator. A lower value of this measure indicates a higher degree of alignment between the domains. In Table 3, we report dist A between each pair of domains at the f -output space (multi-run statistics corresponding to the models trained in Table 2). Clearly, our approach exhibits a higher degree of alignment between the source domains, as compared to learning source-specific classifiers. This encourages the model to learn domain-agnostic features that are more generalizable to the target domain, resulting in an improved alignment between each source-target pair. This translates to an improvement in the target performance.
b) Using a single classifier head. We also train a model by replacing multiple classifiers having agreement with a single classifier. In this case, we fix the number of iterations to be the same as those obtained from SImpAl 50 . In Table 4, we compare the adaptation results of the single-classifier model, against our approach (with agreement using n d classifiers). Clearly, the performance is better when using multiple classifier heads with agreement. For the single-classifier model, we find that during adaptation the performance reaches a peak and then marginally declines to a lower value before reaching the maximum number of iterations, perhaps due to noisy pseudo-labels [13]. In our approach however, classifier agreement aids in pruning those noisy samples near the decision boundaries and enhances the pseudo-labels. This is the added benefit of using multiple classifiers.
To study the extent of latent space alignment using a single classifier, we measure the Proxy Adistance. In Table 3, we present the dist A values at warm-start for the single classifier model and compare it against the two aforementioned approaches ("with" and "w/o" agreement). Particularly, we find that the dist A values are either similar to the model "with agreement", or lower than the model "w/o agreement", indicating that even a single classifier enables alignment to an extent. This explains the efficacy of self-supervised approaches that use pseudo-labels for training under domain-shift.

Ablations using thresholding schemes
In self-training based approaches, pseudo-labeled samples are usually obtained using confidencethresholding, i.e. those samples exhibiting a confidence above a certain threshold τ are chosen for self-training. However, this results in a sensitive hyperparameter τ which requires labeled target  samples for tuning appropriately. In our framework, we propose w(x t , f, h), defined in Eq. 6 of the paper, as a measure of confidence in pseudo-label prediction for target instances, that results in an easy-to-hard curriculum (Sec. 4.2f in the paper). While our framework supports confidence based thresholding, we do not use it for the main results in the paper since it introduces additional hyperparameters. Here, we present empirical results by incorporating thresholds in SImpAl 50 .
We incorporate two types of thresholds, softmax-confidence based (similar to Saito et al. For the softmax-confidence based threshold (τ ), we follow the method applied in Saito et al. [10], where a prediction is considered confident if at least one of the classifiers exhibit the arg maxconfidence above a threshold τ . Intuitively, self-training with such confident target samples will encourage more target samples to fall in the high confidence region. For the percentile-based bagging scheme, we choose the top γ-percentile (most confident) target samples from D t for pseudo-labeling. At each pseudo-label update, the percentile is increased by γ, i.e. the pseudo-labeled bag progressively grows in integral steps of γ. Both methods are trained for the same number of iterations as SImpAl 50 . Table 5 shows that self-training is sensitive to the confidence threshold τ and bagging percentile γ. Although the adaptation performance is seen to improve using thresholds, the best hyperparameter values are highly dataset specific. This calls for labeled target samples to reliably establish the most appropriate hyperparameter values for the task at hand. The study of automated methods to select threshold hyperparameters using unlabeled target samples would be of interest.

Practical Application: Cross-Domain Image Retrieval
We now demonstrate a practical application of Implicit Alignment. Consider an unlabeled dataset D of images sampled from a target domain. Suppose we wish to retrieve images from this dataset based on some class semantics (for e.g., "retrieve images of objects having wheels"). With an unannotated dataset, this task seems non-trivial. However, we show that this is possible by performing Multi-Source Domain Adaptation on the target dataset D using SImpAl. Considering D as the target domain, we adapt a deep model using labeled source datasets and the unlabeled target data D, under the SImpAl framework, and use this model to measure semantic similarity. We develop a cross-domain image retrieval system where, given a reference image x r , we retrieve semantically similar images from a given set of images D (called the query set). To achieve this, we obtain the nearest neighbors of the reference image x r in the set D, based on the class probability vector p (as obtained in Eq. 1 of the paper). This is shown in Algorithm 1.
Note, Algo. 1 does not require any label information. Thus, the query set D can be unlabeled. We show an example use-case where we consider images in the "Quickdraw" (Qdr) domain of DomainNet [9] as the reference images, while the query set is the unlabeled target domain "Real" (Rel). Fig. 4 shows sample images in the two domains, which exhibit a large domain-shift. We obtain a model adapted to the "Real" target domain (i.e. the model corresponding to →Rel in Table 1). Given a reference image in the Qdr domain, an end-user can retrieve semantically similar images from the unlabeled Rel domain using Algo. 1 above.
In Fig. 5, we present qualitative results, demonstrating the retrieval of images from the Rel domain using randomly selected images from the Qdr domain as reference. Note that, all images in Fig. 5 pertain to the test set of the corresponding domains that the model has not encountered during adaptation. The retrieval process using Algo. 1 above works surprisingly well, yielding qualitatively satisfactory results (annotated with green tick-mark). While there are certain images which yield false retrievals (annotated with red cross-mark), many of those cases have incomprehensible reference image (marked with an orange question-mark) and can be ignored during qualitative evaluation.
Further, we design a tool to retrieve images from a chosen query set, by manually "doodling" class semantics (similar to the images in the Qdr domain). See code implementation for the demonstration. Fig. 6 shows the images retrieved during the demonstration.

Code Reference
Pytorch implementation (with cross-domain image-retrieval demo) can be found on the project page 1 .