{'fraction_non_alphanumeric': 0.03331385154880187, 'fraction_numerical': 0.013383985973115138, 'mean_word_length': 4.962020905923345, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Demographic Bias in Presentation Attack Detection of Iris Recognition Systems

With the widespread use of biometric systems, the demographic bias problem raises more attention. Although many studies addressed bias issues in biometric verification, there is no works that analyse the bias in presentation attack detection (PAD) decisions. Hence, we investigate and analyze the demographic bias in iris PAD algorithms in this paper. To enable a clear discussion, we adapt the notions of differential performance and differential outcome to the PAD problem. We study the bias in iris PAD using three baselines (hand-crafted, transfer-learning, and training from scratch) using the the NDCLD-2013 database. The experimental results points out that female users will be significantly less protected by the PAD, in comparison to males.


I. INTRODUCTION
Biometric recognition systems like face, iris, or fingerprint have recently gained wide deployment within various application fields. This has raised two significant questions. One is concerned with the security of recognition systems against presentation attacks. The other is related to the fairness, i.e. if the system performs the same for every different demographic group.
Demographic bias in face recognition was discussed by [9], [17], [23]. Prior works on this issue indicated that face recognition systems obtained inherently a lower performance for certain demographic groups [9], [17]. For example, females are more challenging to recognize compared to males [17]. However, there are no previous works addressing demographic bias in PAD decisions. As iris recognition has become increasingly popular, e.g., iris recognition technology is deployed on consumer smart phones, we address the demographic bias issues in iris PAD systems in this study.
Howard et al. [9] defined four recognition related terms to facilitate the demographic bias research. Similar to terms described for verification in [9], we adapt and reuse two terms to investigate the demographic bias in PAD systems: 1) Differential Performance represents a difference in the bona fide or attack decision distribution between specific demographic groups independent of any decision threshold, 2) Differential Outcome is defined as a difference in Attack Presentation Classification Error Rate (APCER) or Bonafide Presentation Classification Error Rate (BPCER) rates between different demographic groups relative to a decision threshold. We utilize these terms to demonstrate our experiments and examine the bias in Sec.IV and Sec.V.
This work presents the first analysis of demographic bias in PAD systems (of any modality). Specifically, we study gender demographic bias in both hand-crafted and deep-learning based iris PAD algorithms. This is done under variation of the training data bias scenarios. We point out that the PAD decisions tend to be significantly less accurate for female subjects in most of the experimental setups.
II. RELATED WORK a) Bias in Biometric Systems: Danks et al. [3] defined the five types of algorithmic biases: 1) training data bias, 2) algorithmic focus bias, 3) algorithmic processing bias, 4) transfer context bias, 5) interpretation bias. The study of algorithmic bias in autonomous systems has caused researchers to think about bias in biometric systems. Despite the improved performance of biometric recognition systems such as fingerprint, iris or face, the fairness of the recognition systems is still questionable. The current processes in biometric systems include (among others) recognition, classification and presentation attack detection. Biometric recognition is a technology that can identify or verify a person by using biometric information [12], [13]. For example, facial or iris recognition systems compute similarity score between subjects from images or a sequence of video frames. Classification here refers to the process of assigning demographic or other labels to biometric samples [4]. Presentation attack refers to using a presentation to fool the system in an effort to induce an incorrect decision such as using printed images or display videos. In contrast, presentation attack detection systems are expected to automatically determine such attacks [10], [20].
According to recent studies, biometric recognition and classification systems do not perform equally well on different demographic groups. Several face recognition relevant studies indicated that the female, Black, and younger (18−30y.o.) cohorts are inherently more difficult to recognize [9], [17]. Such phenomena that recognition systems have a lower performance for specific demographic groups, is denoted as demographic bias. The study of demographic bias provides a chance to find the recognition system problems and mitigate the biases. b) Bias in PAD systems: Many studies have already pointed demographic bias in face or fingerprint recognition systems [9], [17], [18], [25] and obtained similar conclusions. Moreover, there is one study that experimentally related the bias to biometric attacks, namely the face morphing attack [24]. Garcia et al. [24] suggested that morphing attacks are much more success for Asian females. However, the authors did not study bias in attack detection performance. Since iris biometric features are deployed in many security scenarios, the decision bias in the processes involved must be analyzed and minimized. Unfortunately, no previous works experimentally addressed the demographic bias in PAD systems in general, or the iris PAD specifically. The primary difficulty is limited data resources. Czajka et al.summarized iris PAD databases in [2]. Parts of the databases cannot be shared due to information protection, others were collected many years ago and are unrealistically simple. The rest of the databases contained almost no demographic information. In Sec.IV, we find that only one database, NDCLD-2013 [16] can be used to investigate the demographic bias, even this database has also an imbalanced number of attack samples of males and females. Despite the limited databases availability, the demographic bias in iris PAD is an open issue and is of great interest. We believe, with the widespread use of iris recognition technology, bias problem needs to be analysed to enable future mitigation efforts.

III. IRIS PRESENTATION ATTACK DETECTION
Iris recognition systems are susceptible to the presentation attacks, like printed iris images, replaying videos, or contact lenses. As a result, numerous iris PAD solutions have been proposed to target iris vulnerability issues. Czajka et al. [2] summarized the state-of-the-art iris PAD algorithms in detecting different categories of attack. Various iris PAD frameworks are grouped into hand-crafted and deep-learningbased methods. Hand-crafted features such as Local Binary Pattern (LBP) [6], [19], [28], Binarized statistical image features (BSIF) as well as their variations have been the research focal points until 2015 and made a remarkable contribution to iris PAD problems. After that, with the rapid development and application of deep learning in multiple domains, especially, computer vision field, neural-network-based PAD methods showed up out of nowhere. However, the deeper the network structure e.g. VGG16 [22], the higher the computational needs. Such algorithms with high computational requirements are hard to deploy in mobile devices. Therefore, Howard et al. [8] released two new MobileNet models: MobileNetV3-Large and MobileNetV3-Small to target high and low resource use cases. We utilize the MobileNetV3-Small structure to be one of our PAD baselines.
To investigate the effects of demographic bias on different types of iris PAD systems. These systems are based on 1) hand-crafted features, 2) transfer learning of a general-purpose pre-trained network, 3) a trained from scratch network with computationally-efficient. We use the following baselines to examine the differential performance and differential outcome: 1) LBP + SVM: LBP [6] has been proposed to address the texture-based classification tasks. We adapt the LBP feature in conjunction with a Support Vector Machine (SVM) [1] using a linear kernel to detect the presentation attacks. The threshold, which determines the label of the iris image, is chosen based on the development subset. Finally, the trained SVM model with a specific threshold is used to make a decision in the testing subset, whether the iris image is bona fide or attack. 2) VGG-16 + PCA + SVM: VGG-16 [

IV. EXPERIMENTAL SETUP
In this experiment, we use the NDCLD-2013 database [16] including textured contact lens attack collected by the University of Notra Dame. The detailed iris image capture information has been published in [16]. Briefly, this database contained 5100 images of 330 subjects, which acquired with a LG4000 and an IrisGuard AD100 iris capture devices. At the time of acquisition, Doyle et al. [16] have divided the database into training and testing subsets, which was guaranteed to have an even distribution of genders and ethnicities. Nonetheless, we demonstrate experiments only in one database and only on the gender cohort. The reasons are: 1) The number of the public iris PAD database is much less than the face and iris recognition databases. Due to the awareness of privacy security, demographic information is more difficult to obtain. To our knowledge, only NDCLD-2013 [16] and NDCLD-2015 [15] database had demographic information. While the NDCLD-2013 included gender, race and age categories, the NDCLD-2015 had gender and eye color information. 2) There are only two female subjects with attack samples in the NDCLD-2015 database. Similarly, the eye color was largely unbalanced in this database. Therefore, we have to disregard the NDCLD-2015 database, though we believe that the impact of eye color bias in iris PAD systems is very interesting and worth the investigation. 3) As shown in Fig. 1 Fig. 1: Subject-specific distribution of the demographic variables by different types of representation in database NDCLD-2013 [16]. BF-NL is the bona fide iris image, BF-SL is the iris image with soft contact lens, which is also reported as bona fide, Attack is the iris image with textured contact lens. It should be noticed that race/ethnicity and age categories are unbalanced. Specially, Black and Asian subjects do not have the attack representation. Hence, we investigate only gender bias in this paper.
Given the limited data resources, we are only able to study gender demographics in iris PAD systems. Tow sub-database LG4000 and AD100 are combined into one database. Traintest-split is defined in [16]. A summary of the training and testing samples for each type of representation can be found in Tab.I. Additionally, we split the training subset into identitydisjoint training (80%) and development (20%) sets. In this study, soft lens iris images are used as bona fide samples.  For gender demographic, two experiments are conducted. Because subjects cannot be balanced in 5-fold, we repeat each experiment 5 times to replace the 5-fold cross-validation and both experiments are subject-disjoint. The first experiment measures the iris PAD performance on males versus females. Specifically, the training and development set contains iris images from both male and female subjects. Then, we separately evaluate males and females set. The result from the first experiment can be found in Tab.II. In addition, we report the attack and bona fide decision distribution in the gender cohort from three iris PAD baselines to analyze the differential performance mentioned in Sec.I (see Fig.2, Fig.3, Fig.4). The second experiment investigates the influence of the training subset on iris PAD performance, which means that we use iris images from one gender as a training and development set. For example, we train a model by using only males data from the training set and compute a threshold by fixing APCER at 0.01% in males development set. Then, this model and threshold are applied separately to males and females testing set. Subsequently, we are able to analyse the outcome differences in the scenarios where one gender is exclusively used for training and threshold assignment. The detailed results of three PAD methods from this experiment can be found in Tab.III, Tab.IV and Tab.V.
The following metrics are used to measure the iris PAD algorithm performance: The APCER, BPCER, and HTER that we report are calculated on the same threshold, which chosen by fixing APCER at 0.01% on the development subset. The APCER and BPCER follows the standard definition presented in the ISO/IEC 30107-3 [11].

V. RESULTS AND ANALYSIS
In this section, we analyse the experimental results, which are described in Sec.IV. First, the difference in bona fide and attack PAD decision distribution between males and females, independent of any threshold, is demonstrated to measure the differential performance. Second, a difference in BPCER and APCER between males and females regarding a specific threshold is discussed to examine the differential outcome.
a) Differential Performance in the PAD decision distribution: The most ideal distribution of PAD decision is that the decision of the attack samples is close to zero and the decision of the bona fide sample is close to one. In this subsection, we represent the PAD decision distributions between gender cohort, which are generated by three algorithms: LBP, MobileNetV3-Small and VGG-16 method. First, Fig.2 shows that most bona fide decisions of males are distributed between 0.7 to 1.0 by LBP method. In contrast, the attack and the bona fide decisions of females are relatively difficult to separate. Second, deep-learning-based algorithms obtain a much better performance compared to the LBP method. As shown in Fig.3 and Fig.4, the attack and bona fide decisions generated by MobileNetV3-Small and VGG-16 baselines are relatively separated. Although both gender cases achieved good results, we find out that there is still a small difference in the PAD decision distribution. By observing the zoomed region (in Fig.  3) in MobileNetV3-Small distribution, we notice that male decisions are denser. Moreover, the female cohort performs slightly worse than males, which is noted by the smaller gap between the attack and the bona fide distribution from VGG-16 (see Fig.4). Together, those evidences point out that female presentation subjects are potentially more difficult to detect by iris PAD systems.  b) Differential Outcome in APCER and BPCER: As shown in Tab.II, both hand-crafted-based and neural-networkbased algorithms performed better on the males than females. This finding is consistent with the results of gender cohort researches in face recognition systems [9], [17]. LBP method performs significantly worse on females. The BPCER of LBP method on females (42.60%) is much higher than on males (21.85%) with the same number of bona fide images in testing subset. MobileNetV3-Small and VGG-16 obtain higher APCER on females, but, the same BPCER on both cohorts. Excluding the gender bias effect, one of the possible reasons is due to a small number of female attack samples in the training data as described in [3]. The results of the second experiment possess more information about the nature of the discrepancy. In most cases, better testing performances are achieved on the trained gender cohort when training exclusively on males or females. In addition, the performance of the LBP method indicates the significantly error reduction when training only on males/females than training on mixed data (males + females) in Tab.II. For example, The HTER on females is increased from 25.53% on mixed training data to 41.58% on training only on females (Tab.III). Hence, the LBP method shows that it more difficult to decide if a female subject is bona fide or attack. Besides, VGG-16 method demonstrate a much worse generalizability on the female cohort (Tab.V). The model, which is trained only by female samples, marks male attack samples as bona fide images. MobileNetV3-Small (Tab.IV) provide the consistent generalizability on the females with VGG-16 method. There is still a possibility that iris PAD error on females can be reduced by merely changing the distribution of the female samples. In general, we notice that different PAD solutions performs significantly better on male samples, even in some cases where females samples are exclusively used for training.

VI. CONCLUSION
In this paper, we address the demographic bias problem in iris presentation attack detection systems. Since no previous work discussed bias in PAD, we adapt two terms used for verification bias (differential performance and outcome) analyze the demographic bias on PAD performance. Two kinds of experiment are designed to investigate demographic bias in    iris PAD accuracy, as well as the impact of the training data bias. The results achieved by three PAD baselines showed significant difference in the performance and outcome of the algorithms between male and female samples in most experimental setups.