{'fraction_non_alphanumeric': 0.03661763561366643, 'fraction_numerical': 0.010039691804809714, 'mean_word_length': 5.036880432229269, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Label-Efficient Domain Generalization via Collaborative Exploration and Generalization

Considerable progress has been made in domain generalization (DG) which aims to learn a generalizable model from multiple well-annotated source domains to unknown target domains. However, it can be prohibitively expensive to obtain sufficient annotation for source datasets in many real scenarios. To escape from the dilemma between domain generalization and annotation costs, in this paper, we introduce a novel task named label-efficient domain generalization (LEDG) to enable model generalization with label-limited source domains. To address this challenging task, we propose a novel framework called Collaborative Exploration and Generalization (CEG) which jointly optimizes active exploration and semi-supervised generalization. Specifically, in active exploration, to explore class and domain discriminability while avoiding information divergence and redundancy, we query the labels of the samples with the highest overall ranking of class uncertainty, domain representativeness, and information diversity. In semi-supervised generalization, we design MixUp-based intra- and inter-domain knowledge augmentation to expand domain knowledge and generalize domain invariance. We unify active exploration and semi-supervised generalization in a collaborative way and promote mutual enhancement between them, boosting model generalization with limited annotation. Extensive experiments show that CEG yields superior generalization performance. In particular, CEG can even use only 5% data annotation budget to achieve competitive results compared to the previous DG methods with fully labeled data on PACS dataset.


INTRODUCTION
Despite the remarkable success achieved by modern machine learning algorithms in visual recognition [17,51,56], it heavily relies on the i.i.d. assumption [53] that training and test datasets should have a consistent statistical pattern. Since machine learning systems are usually deployed in a wide range of scenarios where the test data are unknown in advance, it may inevitably result in serious model performance degradation when there exists a distinct distribution/domain shift [44] between the training and test data.
With awareness of this problem, domain generalization (DG) [5] is introduced to extract domain invariance from multiple wellannotated source datasets/domains and train a generalizable model to unknown target domains. Lots of favorable DG algorithms [6,15,42,46,61,71,73] have been proposed recently, however, these methods may need to be fed with a large amount of labeled multisource data for identifying domain invariance and improving model generalization. It might impede the realization of the DG approaches in many real-world applications where labeling massive data could be expensive or even infeasible. For example, a highly accurate and robust system for the detection of lung lesion images of COVID-19 patients may demand a large number of labeled medical images arXiv:2208.03644v1 from different hospitals as the source data for training [16], but it could be impractical to require numerous experienced clinicians to complete the annotation. Therefore, a dilemma is encountered: The requirements of obtaining massive labeled source data for training a generalizable model may hard to be met in realistic scenarios due to the limited annotation budget. Meanwhile, without sufficient labeled data to provide adequate information of multisource distribution, improving model generalization by identifying and learning the domain invariance is at serious risk of being misled.
To escape from the dilemma, we introduce a more practical task named label-efficient domain generalization (LEDG) to enable model generalization with label-limited source domains as shown in Figure  1. Instead of getting fully labeled data, the LEDG task unleashes the potential of budget-limited annotation by querying the labels of a small quota of informative data, and leverages both the labeled and unlabeled data to improve domain generalization. LEDG permits the learning of generalizable models in real scenarios, but it could be much more challenging. The first challenge is from the clear domain distinctions that could exist in the multi-source data, which constitutes enormous obstacles for selecting the most informative samples and learning adequate information of multi-source distribution. Meanwhile, the second challenge is from the discrepant distributions that labeled and unlabeled data may be subject to, making the simultaneous utilization of them for extracting domain invariance and promoting model generalization extremely difficult.
Active learning (AL) [1,19,24,25,45,57] and semi-supervised learning (SSL) [3,4,50,52] provide possible solutions to the introduced LEDG task. AL aims to query the labels of high-quality samples and SSL leverages the unlabeled data to improve performance with limited labeled data. However, the existing AL and SSL methods mostly depend on the i.i.d. assumption, hence may not be favorably extended to the generalization scenarios under distinct domain shifts. Semi-supervised domain generalization (SSDG) [36,47,59,65,70] tackles domain shift under the SSL setting. But some data directly assumed to be labeled in this task might not be helpful for generalization improvement but could increase the annotation costs. Thus, it is imperative to find a solution to the challenging LEDG task for getting rid of the raised dilemma between domain generalization and annotation costs, realizing more practical training of generalizable models in real-world scenarios.
To address the LEDG task, in this paper, we propose a novel framework called Collaborative Exploration and Generalization (CEG) which jointly optimizes active exploration and semi-supervised generalization. In active exploration, to unleash the power of the limited annotation, we query the labels of the samples with the highest overall ranking of class uncertainty, domain representativeness, and information diversity, exploring class and domain discriminability while avoiding information divergence and redundancy. In semi-supervised generalization, we augment intra-and interdomain knowledge with MixUp [67] to expand domain knowledge and generalize domain invariance. An augmentation consistency constraint for unlabeled data and a prediction supervision for labeled data are further included to improve performance. We unify active exploration and semi-supervised generalization in a collaborative way by repeating them alternately, promoting closed-loop mutual enhancement between them for effective learning of domain invariance and label-efficient training of generalizable models.
Our contributions are listed in the following. (1) We introduce a more practical task named label-efficient domain generalization to permit generalization learning in real-world scenarios by tackling the dilemma between domain generalization and annotation costs.
(2) To solve this challenging task, we propose a semi-supervised active learning-based framework CEG to unify active query-based distribution exploration and semi-supervised training-based model generalization in a collaborative way, achieving closed-loop mutual enhancement between them. (3) Extensive experiments show the superior generalization performance of CEG, which can even achieve competitive results using 5% annotation budget compared to the previous DG methods with full annotation on PACS dataset.
Semi-Supervised Domain Generalization (SSDG) [36,47,59,65,70] aims to reduce the reliance of DG on annotation via pseudolabeling [59], consistency learning [70], or bias filtering [65]. For example, StyleMatch [70] combines consistency learning, model uncertainty learning, and style augmentation to utilize the annotation for improving model robustness. However, partial samples assumed to be labeled in the SSDG task may not be informative for boosting model generalization but increase the annotation costs.
Semi-Supervised Learning (SSL). SSL [3,4,23,50,52] is a practicable way to use both labeled and unlabeled data. For example, MeanTeacher [52] achieves significant performance by using the labeled data to optimize a student model, the prediction of which is constrained to be consistent with the prediction of a teacher model. But most of the SSL methods rely on the i.i.d. assumption, which can impair their generalization performance under domain shift.  Active Learning (AL). AL [1,19,24,25,45,57] aims to select high-quality data for querying the labels. Pool-based AL [1,19,25,45] is the most popular that chooses samples from an unlabeled pool and hand them over to the oracle to label. The labeled samples are then added to a labeled pool as newly acquired knowledge. Some successful uncertainty [1,24,57] and diversity [1,45] based methods select uncertain and diverse samples for learning task boundary and comprehensive information, respectively. However, the AL algorithms are mainly designed for single-domain data, thus may not be directly extended to the generalization scenarios.

METHOD 3.1 Label-Efficient Domain Generalization
We begin with the task setting of the introduced Label-Efficient Domain Generalization (LEDG). In the LEDG task, we have unlabeled source datasets {D 1 , ..., D } sampled from different data distributions { ( 1 , 1 ), ..., ( , )}, respectively. There are unlabeled data points being sampled for each dataset D , i.e., D = { } =1 , for = 1, ..., . We further have an annotation budget , i.e., the maximum number of samples that we are allowed to query their class labels. Each sample pair ( , ) is defined on the image and label joint space X × Y. Besides, the domain label for each sample is given in our task. We consider a classification model composed of a feature extractor and a classifier head , i.e., = • . The goal of LEDG is to train the model by utilizing the unlabeled multi-source data {D } =1 as well as the limited annotation budget for improving the generalization performance of the model on the target domains with unknown distributions. For convenience of the statement of our method, we denote the dataset consists of all the labeled (queried) samples ( as are the data size of the labeled and unlabeled datasets, respectively. It is obvious that the whole data size ( ) + ( ) = 1 + 2 +...+ . Our insight for this challenging task is to consider the labeled and unlabeled samples as the "known" and "unknown" regions of the multi-source distribution, respectively. In view of this, the core idea of our solution is to: (1) explore the key knowledge hidden in the unknown regions via active query for adequate multi-source distribution learning, (2) extract and generalize the domain invariance contained in the obtained knowledge in both the known and unknown regions via semi-supervised training, and (3) make the active query-based exploration and semi-supervised training-based generalization complement and promote each other to train a generalizable model. An overview of our framework, i.e., Collaborative Exploration and Generalization (CEG), is shown in Figure 2.

Active Exploration
There might be distinct domain divergence among the data distributions of the source domains. Meanwhile, each source domain contains the discriminative information of class boundary which is essential for the prediction task. Thus, we take class and domain discriminability as the key knowledge for learning the multi-source distribution in the active exploration. In light of this, we present to select the samples with high class uncertainty and domain representativeness. To avoid information redundancy, we further take information diversity into consideration. Figure 3 provides revealing insights into the elaborate strategy for the active exploration.
To capture the key knowledge of class discriminability, we select the samples with high class uncertainty. Specifically, we adopt the margin of the top two model predictions to choose class-ambiguous samples to query. Let ℎ be the ℎ-th dimension of the class prediction of the model (after softmax operation), then the class uncertainty score for each unlabeled sample ( ) is defined as (1) We tend to query the samples with high uncertainty scores for their class-ambiguous. Labeling these samples provides the key knowledge of class discriminability, which helps the model to figure out class boundary and boosts its performance of class prediction. Different from the single-domain scenario considered in the AL methods [24,57], multiple source domains may lead to an information divergence problem here, i.e., the selected high-uncertainty samples are scattered at the domain boundary (see Figure 3 (a)).

Samples in different domains
To sufficiently explore and grasp information of the multi-source distribution, the selected samples are required to represent the characteristics of each source domain. Therefore, given domain label ( ) of each unlabeled sample ( ) , we first train a domain discriminator model with a domain discriminability loss: where ℓ is the cross-entropy loss. Let ℎ be the ℎ-th dimension of the domain prediction of the model , we then define domain representativeness score for each unlabeled sample ( ) as: Note that different from the class discriminability learning with class ambiguous data, here, we select the samples with high representativeness score, i.e., high domain confidence. It prevents the model from learning class discriminability in the remote areas of each source domain and hence loosing domain characteristics. Then an information redundancy problem has arisen, i.e., the selected samples with high class uncertainty and domain representativeness may gather together (see Figure 3 (b)), which wastes the limited annotation budget. To disperse the information, we choose the samples that are far away from the known domain-class knowledge of the labeled data. We make a knowledge dataset K ℎ be composed of the labeled data ( ( ) , ( ) ) ∈ D ( ) that belongs to domain and class ℎ (if there is no such a sample in D ( ) , then K ℎ = ∅). Let |K ℎ | be the number of samples in the knowledge dataset K ℎ and be the feature extractor. We generate knowledge centroids ℎ for the known regions in the semantic feature space: We let a set B be composed of all the knowledge centroids ℎ if K ℎ ≠ ∅. Then, we define information diversity score as where dist(·, ·) is a distance metric used as cosine distance in experiments. We tend to choose samples with high diversity score, i.e., far away from the closest centroids, facilitating the unknown exploration for comprehensive learning of multi-source distribution.
To avoid numerical issues, we integrate uncertainty score , representativeness score , and diversity score by adopting their rankings, which we denote as ′ , ′ , and ′ , respectively. Finally, we have an overall query ranking for each unlabeled sample ( ) : where 1 and 2 are trade-off hyper-parameters. Note that we rank each score, i.e., , , and , from high to low, for selecting the most informative samples of the multi-source data distribution.

Semi-Supervised Generalization
With active query-based exploration, we have a small quota of labeled data and massive unlabeled data, i.e., small range of known regions and large range of unknown regions of the data distribution. In semi-supervised generalization, we aim to expand domain knowledge and learn domain invariance via MixUp-based intraand inter-domain knowledge augmentation as shown in Figure 4. We start by defining the unlabeled samples that close to the knowledge centroids ℎ as "reliable samples", and construct a reliable dataset U with expansion threshold to tune reliable range: where pseudo label ( ) of each unlabeled sample ( ) is assigned by the nearest knowledge centroids ℎ , that is, ( ) = arg min ℎ: A low threshold value leads to few reliable samples but high dependability of its pseudo labels, and vice versa. To arrange learning tasks in the order of difficulty for helping the model gain sufficient basic and easy knowledge before handling more complex data, we let increase with epochs and dynamically tune learning difficulty: where and are the initial and final threshold values, and are the total and current epochs, respectively. It makes the model expand knowledge stably with the high dependable samples at the beginning, and break through the hard samples gradually.
We expand domain-class knowledge in each domain and across domains with the reliable dataset U, and construct MixUp-based intra-and inter-domain knowledge augmentation datasets, i.e., M and M , respectively. That is, where ∼ ( , ) with = 0.2 as in [67].
We further utilize the unknown and known regions by adopting augmentation consistency constraint [50] for the unlabeled data and prediction supervision for the labeled data, respectively. Let A and A be weak (flip-and-shift) and strong [11] augmentation functions, respectively. Pseudo labelsˆcan be assigned by A viâ = arg max ℎ ℎ (A ( ( ) )). The augmentation consistency loss

Algorithm 1 Collaborative Exploration and Generalization
Input: Source datasets {D } =1 ; Pretraining epochs and learning epochs ; Annotation budget and initial budget . Output: A well trained prediction modelˆ; 1: Initialize labeled dataset D ( ) with ; 2: for = 1 to do 3: Get knowledge centroids ℎ with D ( ) via Equation (4); 4: if > then // begin to query after pretraining 5: Select a subset ΔD ( ) from D ( ) via Equation (6) Train the discriminator with D ( ) via Equation (2); 10: Train the model with D ( ) and D ( ) via Equation (15). 11: end for L makes the model predictions of strong augmented data, i.e., (A ( ( ) )), and weak augmented label, i.e.,ˆ, to be consistent: where an indicator ( ( ) ) = 1(max ℎ ( ℎ (A ( ( ) )) ≥ ) ( is set to 0.95 as in [50]) selects high dependable data. This constraint helps the model to capture structural knowledge in the unknown regions via unsupervised learning. For prediction supervision, we adopt a cross-entropy classification loss L for the labeled data: A semi-supervised training loss L is then derived as: where is a hyper-parameter of knowledge expansion and generalization. We set the weights of L and L to 1 as in [50]. Our framework CEG explores informative unlabeled samples for learning key knowledge of multi-source distribution using limited annotation budget, promoting the expansion and generalization of the key knowledge in semi-supervised training. Then a well trained model is continuously utilized to select more effective samples in the next round of query. The active exploration and semi-supervised generalization are unified in a collaborative way by being repeated alternately. They complement and promote each other to enable label-efficient domain generalization. The learning process of CEG is stated in Algorithm 1. Note that we use an initial budget from the annotation budget to initialize the labeled dataset ( ) via uniform sample selection, and pretrain the model before active query to solve a cold start problem, to our empirical experience.

EXPERIMENTS
In this section, we first evaluate our framework CEG in labellimited scenarios, and then give sensitivity analysis of hyper-parameters, ablation studies of the components, and in-depth empirical analysis.
Implementation details. Following [6,21,61], we use a pretrained ResNet-18 [17] as the backbone and conduct leave-onedomain-out experiments by choosing one domain to hold out as the target domain. For fair comparisons, we implement all the methods with the same settings, i.e., SGD optimizer with learning rate 0.003 for feature extractor and 0.01 for classifier, pre-training/learning epochs on PACS and Office-Home datasets are 30/30 and 15/15, respectively, and batch-size is 16, et al. In experiments, we directly use " " to represent " " in Equation (9) for simplicity. We adopt   the percentage of the unlabeled samples for instead of a distance value. We set = 2 . The hyper-parameters { , , 1 , 2 } are set to {0.3, 50%, 3, 1} and {0.1, 70%, 0.5, 0.5} for PACS and Office-Home, respectively. Half annotation budget is used as the initial budget to initialize the labeled dataset. We report the results over five runs.

Main Results of CEG
CEG vs DG methods. Table 1 and 2 report the results with 5% annotation budget on PACS and Office-Home datasets, respectively. We observe that the accuracy of DG methods drops rapidly when only 5% labeled data is given. In comparison, our method CEG can select the informative data to label and utilize both the labeled and unlabeled data to boost generalization performance in this challenging label-limited scenario. Most notably, CEG can even achieve competitive results with only 5% annotation budget compared to the DG methods with full annotation on the PACS dataset.
It reveals that CEG generally realizes label-efficient domain generalization by exploiting only a small quota of labeled data and massive unlabeled data. We attribute this success to the effective collaboration mechanism between active exploration and semi-supervised generalization, which unleashes the latent power of the limited annotation budget. Since the DG methods may not be good at tackling the label-limited task as they can only use the labeled data, we further compare our CEG method with AL, SSL, and SSDG methods.
CEG vs AL, SSL, SSDG methods. Table 3 reports the results with 5% annotation budget on PACS and Office-Home datasets. CEG outperforms other methods on half of the tasks and yields the best average accuracy on the PACS dataset. It is probably because the AL and SSL methods rely on the i.i.d. assumption, and the SSDG method does not label and exploit the important source data. In contrast, CEG selects the most informative samples for query via active exploration, and hence captures multi-source distribution and boosts generalization ability more accurately. Besides, the performance of CEG is significantly better than other methods on the Office-Home dataset. We attribute it to the construction of domain-class knowledge centroids, which greatly helps CEG to precisely explore unknown regions during active exploration, and effectively expand knowledge and generalize domain invariance during semi-supervised generalization on the Office-Home dataset (because Office-Home has 65 classes but PACS only has 7 classes).

Sensitivity Analysis
As shown in Figure 5, CEG is generally robust to the hyper-parameters and outperforms other methods even with the default settings, i.e.,  on PACS and 50.37% on Office-Home), indicating that exhaustive hyper-parameter fine-tuning is not necessary for CEG to achieve excellent performance in label-efficient generalization learning. Figure 6 shows the results with increasing annotation on Office-Home dataset. CEG consistently outperforms other methods by sharp margins on the average accuracy and three of the four tasks. The significant performance achieved by CEG when given a low budget is probably due to the query based-active exploration, but this advantage could be weakened when given a higher budget.

Why does CEG Work?
Ablation studies are reported in Table 4. The three criteria of active exploration, i.e., uncertainty , representativeness , and diversity , are all important for learning multi-source distribution, and the integration of them further make full use of the limited annotation, compared with uniform selection. For semi-supervised generalization, both knowledge expansion and generalization L and augmentation consistency L are necessary to yield remarkable results. The intra-and inter-domain knowledge augmentation datasets, i.e., M and M , both play vital roles in improving generalization performance. It is noteworthy that the proposed L significantly improves average accuracy from 42.27% to 50.87% on Office-Home. Besides, the devised dynamic threshold shows its effectiveness of learning with increasing difficulty compared to the static one. The above results illustrate that each component is indispensable, and the exploration and generalization complement and promote each other for achieving the excellent performance.
T-SNE visualization is shown in Figure 7. The left figure shows class-ambiguous samples, i.e., the samples distribute on class boundary, are selected for learning class discriminability. The right figure shows that, in general, the selected samples distribute uniformly and representatively in each domain, illustrating the effectiveness of the domain representativeness and information diversity criteria. These three criteria help CEG to select the most informative samples for learning multi-source distribution, which facilitates the generalizable model training in semi-supervised generalization.
Accuracy curve on Office-Home dataset is shown in Figure 8. Active exploration selects the most important samples and grasps the key knowledge of multi-source distribution to effectively improve performance on each domain, compared with uniform sample selection. Semi-supervised generalization further markedly boosts performance by expanding the obtained knowledge and generalizing domain invariance. They promote each other to achieve remarkable generalization performance on the target domain.

CONCLUSION
We introduce a practical task named label-efficient domain generalization, and propose a novel method called CEG for this task via active exploration and semi-supervised generalization. The two modules promote each other to improve model generalization with the limited annotation. In future work, we may extend our method to a more challenging setting that domain labels are unknown.