{'fraction_non_alphanumeric': 0.030678938241963452, 'fraction_numerical': 0.007869814592503669, 'mean_word_length': 5.075634792004322, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Dual Reweighting Domain Generalization for Face Presentation Attack Detection

Face anti-spoofing approaches based on domain generalization (DG) have drawn growing attention due to their robustness for unseen scenarios. Previous methods treat each sample from multiple domains indiscriminately during the training process, and endeavor to extract a common feature space to improve the generalization. However, due to complex and biased data distribution, directly treating them equally will corrupt the generalization ability. To settle the issue, we propose a novel Dual Reweighting Domain Generalization (DRDG) framework which iteratively reweights the relative importance between samples to further improve the generalization. Concretely, Sample Reweighting Module is first proposed to identify samples with relatively large domain bias, and reduce their impact on the overall optimization. Afterwards, Feature Reweighting Module is introduced to focus on these samples and extract more domain-irrelevant features via a self-distilling mechanism. Combined with the domain discriminator, the iteration of the two modules promotes the extraction of generalized features. Extensive experiments and visualizations are presented to demonstrate the effectiveness and interpretability of our method against the state-of-the-art competitors.


Introduction
Face recognition techniques [Schroff et al., 2015;Deng et al., 2019] have been widely utilized in our daily lives, such as smart device authentication, access control and entrance guard systems. Although bringing convenience, it also raises public concern about the safety issue since many kinds of face presentation attacks appear, such as photos and videos, which can easily fool the face recognition systems. Therefore, many hand-crafted features based and deep learning based methods for presentation attack detection have been proposed in recent years. Hand-crafted methods utilize traditional feature descriptors, such as Local Binary Pattern * Equal Contribution † Corresponding Author Figure 1: Previous face anti-spoofing methods treat each sample from multiple domains indiscriminately during the training process. While our method iteratively reweights the relative importance between samples to further improve the generalization.
Although the above state-of-the-art methods have attained remarkable performance in the intra-dataset setting where training and testing data come from the same distribution, the performance drops significantly encountering the testing data from different distributions. This is because the above methods don't take domain discrepancy into account, resulting in extracting biased features. To overcome the limitation, some researchers introduce the domain generalization (DG) methods for face presentation attack detection, which utilize several source domains to improve the generalization abilities of the model. Based on it, existing methods [Shao et al., 2019;Jia et al., 2020] attempt to align the features of multiple source data equally for a shared feature space. However, for some samples are little domain-biased and some ones are large domain-biased, training on such samples indiscriminately will corrupt the generalization abilities of the methods. Hence, how to alleviate such affection to learn a more gener-alizable representation remains a challenging problem.
To settle the problem, on one hand, inspired by curriculum learning, we argue that focusing on large domain-biased samples from the beginning will affect the overall optimization. However, on the other hand, solving these large domainbiased samples can further improve the generalization of the model. In order to stabilize the overall optimization direction and improve generalization, we propose a novel Dual Reweighting Domain Generalization (DRDG) framework for face anti-spoofing by iteratively reweighting the relative importance between samples, as illustrated in Figure 1. Specifically, we propose two reweighting modules, Sample Reweighting Module (SRM) and Feature Reweighting Module (FRM). Firstly, Sample Reweighting Module tries to identify samples with relatively large domain bias and reduce their impact on overall optimization. Secondly, Feature Reweighting Module (FRM) specially focuses on these large domainbiased samples and tries to extract more domain-irrelevant features via a self-distilling mechanism, which will promote generalization abilities. In general, these two modules are updated iteratively relying on the competition against the domain discriminator. And the mutual iteration of these two modules ensures the overall optimization is not excessively affected by the large domain-biased data, while at the same time, further distills extracted features, and finally achieves better generalization.
The main contributions are summarized as follows: • We propose a novel Dual Reweighting Domain Generalization framework for face anti-spoofing, which iteratively reweights the relative importance between samples from multiple source domains to further improve the generalization.
• We specially design Sample Reweighting Module (SRM) and Feature Reweighting Module (FRM) to ensure that the overall optimization direction is not excessively affected by the large domain-biased data, and at the same time, pay more attention to distilling the feature of these data, resulting in the more generalizable model.
• Abundant experiments and visualizations are presented, which demonstrates the effectiveness of our method against the state-of-the-art competitors.

Face Anti-Spoofing
In recent years, researchers have made great progress in the face anti-spoofing area. The development of it is concluded into two stages. Early researchers mainly utilized handcrafted feature descriptors, such as LBP [Boulkenafet et al., 2015], HOG [Komulainen et al., 2013], SIFT [Patel et al., 2016] and trained a traditional classifier for judgment. Some methods focused on temporal features to detect movement, such as eye blinking [Pan et al., 2007], head motion and lips motion. With the rise of deep learning, [Feng et al., 2016] regarded the face anti-spoofing as a binary classification task and leveraged the CNN architecture to solve it. To further boost the performance, [Liu et al., 2018] utilized additional supervisions, such as depth map, reflect map and r-ppg signal, to boost the performance. Based on auxiliary information, Liu et al., 2020] regularized features from the perspective of disentanglement. Since the above methods did not pay attention to the cross-dataset setting, MADDG [Shao et al., 2019] utilized multiple domain discriminators to learn a generalized feature space. While SSDG [Jia et al., 2020] only aligned the features of real samples from different datasets but not for the features of the fake ones. Moreover, [Chen et al., 2021] proposed D 2 AM to settle a more challenging generalizable scenario in the real world where domain labels are unknown. However, these DG-based methods all attempt to align the samples straightforwardly and uniformly to a common feature space, regardless of the complexity and bias of data distribution, which leads to the degradation of generalization.

Domain Generalization
Domain generalization aims to extract knowledge applicable to any unseen domain with multiple source domains, when target domain information is not available at training time. Several domain generalization methods have been proposed recently. [Ghifary et al., 2015] matched the feature distributions among source domains by using an auto-encoder. The work in [Li et al., 2018b] based on adversarial learning to match the multiple domains to a pre-trained distribution. MLDG [Li et al., 2018a] proposed a model-agnostic meta-learning for domain generalization. [Seo et al., 2020] utilized batch and instance normalizations to achieve the best combination for each domain. Based on the above DG methods, some face anti-spoofing methods have been proposed. However, they directly treat these samples equally, which corrupts the generalization ability. While our framework, Dual Reweighting Domain Generalization (DRDG), iteratively reweights the relative importance between samples to further improve the generalization. , which reassign the most suitable weights to samples and features separately, leading to a discriminative and compact feature space. Moreover, we introduce the iterative optimization strategy, which further facilitates the cooperation of SRM and FRM for the better generalization on the unseen target domains.

Sample Reweighting Module
Since some samples from multiple source domains contain large distribution discrepancies, aligning them at the early training stage inevitably introduces convergence problems and hinders the model's generalization ability. To this end, we propose to dynamically adjust weights for different samples and reduce the impact of large domain-biased samples on the overall optimization. In our assumption, the samples, which are difficult to be distinguished for the domain discriminator, are the small domain-biased ones. On the contrary, the easily distinguished samples are the large domain-biased ones. Based on the assumption, we introduce a novel module Sample Reweighting Module (SRM) to assign weights for each sample via the adversarial procedure against the domain discriminator (described in Sec. 3.4). Then, we utilize these weights to encourage the framework seeking a generalized feature space by bringing the distributions of high-weight samples much closer to each other and avoiding the influence of low-weight samples.
Particularly, SRM is responsible for assigning the weights to samples at each iteration, maximizing the loss of the domain discriminator. To keep up with the updated domain discriminator, SRM is forced to update constantly and reassign the weights across training. Considering the distribution discrepancies between the live and fake faces, we introduce two SRM modules, denoted as SRM real and SRM f ake , to reweight the samples separately for each class, which are denoted as SRM uniformly because of the same mechanism. SRM assigns weights to each sample x i , which is denoted by W i . Given a batch of samples, x 1 , x 2 , ...x N , the number of samples in each domain is the same. SRM maps the features of these samples to an array of scores normalized via sigmoid function as the reweighting vector. During training, the optimization objective of SRM is formulated as Equation.2, where F F RMi is the feature map obtained from FRM, D is Discriminator, N is the number of one batch samples and d i

Feature Reweighting Module
To make sure that our framework tackles the large domainbiased samples to facilitate learning a more generalizable representation finally, we propose the Feature Reweighting Module (FRM) through a self-distillation method to achieve this. Specifically, this module acts as an information modulator to self-distill common information from large domain-biased samples further, automatically selecting the proper feature channels to modulate contained information. The modulated features amplify the domain-agnostic information and lessen the domain-relevant information.
Following the hypothesis in Sec. 3.2, FRM employs it to self-distill the information of each sample via selecting the channels with different weights. While interplaying with domain discriminator, FRM utilizes reverse weights 1 − W i to aggregate the large domain-biased samples much more for gradually tackling these samples, without impacting on other modules. These two modules, FRM and SRM, facilitate each other to extract a robust representation, boosting the generalization performance. On one hand, FRM relies on the sample weights calculated from SRM. Hence FRM is updated after SRM in each iteration to fully utilize the domain-biased samples currently. On the other hand, FRM suppresses the domain-relevant information of large domain-biased samples to help SRM out of the trap, improving the relative importance of such samples.
Concretely, the inputs of FRM are the features F ∈ R N ×H×W ×C extracted by Feature Encoder, where N is the number of images in each mini-batch, H × W is the spatial size of features and C is the dimension of features. To ensure that each dimension is scaled with different scales, FRM calculates the weights for them via softmax function. During training, we minimize the reweighted loss of the domain discriminator L F RM to obtain the weight A i as follows: where F i is the feature map from Feature Encoder, D is the Discriminator, N is the number of samples in one mini-batch and d i is the domain label of image x i , A i is the channelwise weight of feature F i , W i is the weight of the sample x i , θ F RM is the parameters of FRM, ⊗ denotes the element-wise multiplication.

Iterative Learning Framework
In this section, we briefly introduce other modules and the optimization strategy of our framework, as shown in Algorithm 1. Following the conventional DG-based methods for face anti-spoofing, we utilize Feature Encoder, Binary Classifier, Depth Estimator and Domain Discriminator in the framework. Concretely, Feature Encoder is proposed to extract domain-invariant features for better generalization on the unseen domains, which is optimized via L Enc . Binary Classifier aims to detect the attacks from the real faces, which is optimized via L W Cls . Depth Estimator is introduced to estimate the facial depth map to guide the learning of the encoder. The pseudo-depth maps as grounding truth are produced by PRNet [Feng et al., 2018]. Depth Estimator is optimized via L W Dep . Domain Discriminator is utilized to judge whether the extracted features are sufficiently domaininvariant, which is optimized via L Dis . And we only utilize the output of the classifier as the final results for face presentation attack detection. To clearly explain the iterative learning process, here we describe one complete iterative unit in detail.
Step A. We update the Discriminator (D) with the discriminator loss L Dis firstly, because the two modules SRM and FRM compete against it, which requires the discriminator good enough to distinguish the domain of each sample.
Step B. Then SRM is utilized to estimate weights W of each sample for focusing on the small domain-biased ones. Combining W and the discriminator loss L Dis , we update SRM to minimize the loss L SRM , which improves the estimation abilities of SRM for more accurate reweighting.
Step C. Next step, the weights W are utilized to calculate other loss functions, L W Dep , L W Cls and L Enc . We update Depth Estimator (Dep), Binary Classifier (BC), Feature Encoder (Enc) and (FRM) with the above loss.
Step D. Repeating the above steps K-1 times, we utilize FRM to handle the large domain-biased samples and extract more generalizable representation further via the self-distillation mechanism. Then, we update FRM only by the loss L F RM , which is calculated by the discriminator loss L Dis and the reverse weights 1 − W of the corresponding iteration.
Update θ D with L Dis ; 10 else 11 Calculate L SRM as Eq. 2;

12
Update θ SRM with L SRM and update W ;  [Shao et al., 2019;Shao et al., 2020;Jia et al., 2020], utilizing four public databases, to evaluate our method: OULU-NPU (denoted as O) [Boulkenafet et al., 2017], CASIA-FASD (denoted as C) [Zhang et al., 2012], MSU-MFSD (denoted as M) [Wen et al., 2015] and Idiap Replay-Attack (denoted as I) [Chingovska et al., 2012]. Concretely, we select one dataset for testing and the remaining three for training, then four testing tasks are obtained: O&C&M to I, O&C&I to M, O&M&I to C and I&C&M to O. Since significant domain shift exists among these datasets, e.g., materials, illumination, background, resolution and etc., the tasks are challenging. Implementation Details. Our method is implemented via PyTorch on 11G NVIDIA 2080Ti GPUs with Linux OS and trained with Adam optimizer [Kingma and Ba, 2014]. The learning rates α, β are set as 1e-3, 1e-4, respectively. We extract RGB and HSV channels of images, thus the input size is 256 × 256 × 6. In training phase, the balance coefficients λ 1 and λ 2 are set to 10 and 0.1 respectively. In our method, the K in Algorithm 1 is an important hyperparameter, which determines the training pace of large domain-biased samples, and we set it as 5 according to our experiments. We strictly follow the popular evaluation metrics, which contain the Half Total Error Rate (HTER) and the Area Under Curve (AUC).

Comparison Results
As shown in Table 1 and Figure 3, our method outperforms all state-of-the-art methods. Firstly, we can observe that the DGbased face anti-spoofing methods perform better than conventional methods. This proves that the distribution of target domain is different from source domain, while conventional methods only focus on the spoofing cues fitting source domain. Moreover, the proposed method outperforms other DG-based methods. Concretely, compared to MADDG [Shao et al., 2020], DRDG gains 5.26% and 5.45% improvement on the task O&C&I to M and O&M&I to C respectively with HTER. We believe the reason is that our method focuses on small domain-biased samples and self-distill the common information from large domain biased ones, the cooperation of which facilitate the generalization. As shown in Table 2, we also evaluate our method with extremely limited source domains, and it achieves the best performance, which verifies its effectiveness again. As for computational efficiency, since SRM is discarded and only FRM is retained during the inference stage, our method is comparable to others.

Ablation Study
We perform the ablation study to verify the contribution of each component on O&C&I to M task. As shown in Table 3, several observations are made as follows: 1) Compared with Baseline where both SRM and FRM are removed from DRDG, Baseline FRM gains a better performance, which demonstrates that FRM enhances the features via a selfdistillation mechanism to extract a more common feature for generalization. Moreover, Baseline FRM reverse aims to push the samples from different domains apart, which performs even worse than Baseline. 2) Baseline SRM performs better than Baseline, while Baseline SRM reverse which emphasizes large domain-biased samples first and small domainbiased ones later performs worse than it. It proves the correctness of our training strategy via starting from the small domain-biased ones than dealing with the large domainbiased ones. 3) DRDG achieves the best performance compared to the other variants, which confirms that FRM and SRM facilitate each other to extract a robust representation and boost the generalization performance.

Visualization and Analysis
Sample Reweighting Visualization. To verify the effectiveness of SRM, we randomly select a model at a certain epoch during training for visualization, which is conducted on O&C&I to M task. We first demonstrate some high-weight and low-weight samples assigned by SRM in three source domains, which are from the same mini-batch, as shown in Figure 4(a). It is obvious that the differences between lowweight samples are significantly larger than those between high-weight ones. It is attributed to their large distribution discrepancies, e.g., sample with overexposed light and one with reddish overall style (all marked with red boxes). Focusing on them may mislead the optimization of the network, hence SRM assigns the lower weights to them, alleviating their impact. As shown in Figure 5(a), We also visualize the feature distributions of these samples via t-SNE. We find that high-weight samples aggregate much more closer to each other and low-weight samples are much far away from the center like outliers. Aligning these samples equally corrupts the generalization ability of our framework. Hence, SRM indeed ensures that our framework automatically selects the most suitable samples for alignment in each mini-batch, facilitating the overall optimization for better generalization.
Feature Reweighting Visualization. Following the experiments of SRM, we utilize the same model to draw the visualization for FRM. To make an exploration about the channels with different weights, we demonstrate the feature maps of the low-weight samples selected from SRM, as illustrated in Figure 4(b). We find that the attention maps of low-weight channels are with low energy and focus on some trivial cues, such as backgrounds, clothes, etc, which are not generalizable. While attention maps of high-weight channels are always with high energy and mainly pay attention to the region of the internal face, which are more likely to be intrinsic and generalized. Hence the self-distillation mechanism of FRM indeed promotes the extraction of common features for better generalization abilities. The distribution of the features extracted by the updated FRM is also illustrated in Figure 5(b). Compared with Figure 5(a), we find that the large domainbiased samples are pulled back to the center after the update of FRM, which verifies that FRM tackles such samples to get a more compact feature space. Feature Distribution. To study the effect of the framework containing two modules on the feature distributions, we draw the visualization of Baseline and framework DRDG, as shown in Figure 5(c),(d) separately. Since baseline tries to align all the samples regardless of their distribution discrepancies, the samples relatively scatter in the feature space, which results in the performance degradation on the target domain. While feature space constructed by framework DRDG is more compact than Baseline overall, it achieves better class boundary and generalizes well on target domain for two modules facilitating each other.

Conclusion
In this paper, we propose a new perspective of FAS that automatically selects the most suitable samples for domain generalization. Concretely, we present an iteratively learning framework, Dual Reweighting Domain Generalization, containing two novel modules, to boost the generalization abilities. SRM and FRM facilitate each other to establish the training strategy via reassigning the most suitable weights to samples and features respectively, and pull all the samples aggregated for better generalization results. Comprehensive experiments and visualizations are presented to demonstrate the effectiveness and interpretability of our method against the state-of-the-art competitors.