{'fraction_non_alphanumeric': 0.03390144497962208, 'fraction_numerical': 0.014608585190282115, 'mean_word_length': 5.024713010204081, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Accuracy comparison across face recognition algorithms: Where are we on measuring race bias?

Previous generations of face recognition algorithms differ in accuracy for faces of different races (race bias). Whether deep convolutional neural networks (DCNNs) are race biased is less studied. To measure race bias in algorithms, it is important to consider the underlying factors. Here, we present the possible underlying factors and methodological considerations for assessing race bias in algorithms. We investigate data-driven and scenario modeling factors. Data-driven factors include image quality, image population statistics, and algorithm architecture. Scenario modeling considers the role of the"user"of the algorithm (e.g., threshold decisions and demographic constraints). To illustrate how these issues apply, we present data from four face recognition algorithms (one pre- DCNN, three DCNN) for Asian and Caucasian faces. First, for all four algorithms, the degree of bias varied depending on the identification decision threshold. Second, for all algorithms, to achieve equal false accept rates (FARs), Asian faces required higher identification thresholds than Caucasian faces. Third, dataset difficulty affected both overall recognition accuracy and race bias. Fourth, demographic constraints on the formulation of the distributions used in the test, impacted estimates of algorithm accuracy. We conclude with a recommended checklist for measuring race bias in face recognition algorithms.


Introduction
Scientists have over 50 years of experience studying the effects of race on human face recognition ability. People recognize faces of their "own" race more accurately than faces of "other" races [1]. This phenomenon is referred to as the "other-race effect" (ORE). The study of race bias in computational algorithms, likewise, has a nearly 30-year history that converges on the following finding. Nearly all of the face recognition algorithms studied over the past 30 years show some performance differences as a function of the race of the face. As race bias is investigated in deep convolutional neural network (DCNN) algorithms, it is important to consider the lessons learned from both humanand machine-based studies of race bias in face recognition over the last many decades [1], [2]. This paper is organized as follows. First, we review the combined effects of subject and race of face on face recognition by humans. Second, we review race bias findings for face recognition algorithms. These findings can guide us as we move from previous-generation (simpler) algorithms to present-day DCNNs. These latter algorithms require massive amounts of training data and employ large numbers of local, non-linear computations. Third, we discuss critical, but often overlooked, considerations in measuring face recognition bias in algorithms. Fourth, we will apply the lessons we have learned from the past to measure performance bias in DCNN-based face recognition systems. Specifically, we consider data from three DCNNs and one pre-DCNN algorithm. We conclude with a checklist and discussion of considerations to bear in mind when assessing bias in algorithms. Our goal is to better understand how to measure and interpret race bias in face recognition algorithms.

Other-race effect -Humans
The ORE for humans has been found across multiple racial/ethnic groups using different methodical paradigms [1], [3]. This effect is measured in an experimental paradigm, whereby subjects of different races are tested on their ability to recognize faces of two (or more) races. Formally, the effect is defined by a statistical interaction between the race of the subject and the race of the face that shows a relative accuracy advantage for own-race faces.
Previous research has demonstrated that expertise in recognizing faces comes, in part, from meaningful experiences with the faces across the lifespan [4], beginning in infancy [5], [6]. Indeed, race biases in face identification have been found across different age groups [7], [8], [9]. Thus, psychologists have concluded that the power to recognize faces with high accuracy comes from experience discriminating among individuals from an homogeneous population of highly similar faces (i.e., faces of one's own race) [3], [4], [10], [11], [12]. This experience enables the development of neural features that maximize encoding differences among faces within the group. By this account, for humans the ORE is due to the fact that we represent faces of our own-race with a highly effective specialized set of features. These features are not well suited to encode the unique character of other-race faces. Consequently, the experience-dependent nature of human expertise for faces is both a strength and weakness of the human perceptual system. Experience helps us to tune our perceptual systems to rely on features that are optimally suited to faces of our own race, but at the cost of identification performance for faces of other races.
Lesson 1. The fact that the ORE has been replicated across multiple races of subjects and faces, combined with findings that experience affects face recognition accuracy in predictable ways, leads us to two conclusions. First, human face recognition ability is characterized by an other-race effect, whereby performance is relatively more accurate for faces of one's own race. Second, the psychological findings support the assumption that faces of all races should be theoretically equally recognizable, if one applies appropriate features to analyze a particular race of faces.

Racial Bias -Algorithms
Given that humans show an other-race effect for face recognition, it is not surprising that previous research has investigated the effects of race on the accuracy of face recognition algorithms [13], [14], [15], [16], [17], [18], [19]. For obvious reasons, these effects focus generally on race bias (accuracy differences as function of stimulus race) and not the other-race effect (interaction between the race of the face and race of stimulus). The integration of state-of-theart face recognition algorithms in applied settings (e.g., airports) underscores the importance of measuring the effects of race on the accuracy of face recognition algorithms. In what follows, we divide this review of race bias in face recognition algorithms into studies that examine previousgeneration algorithms and those that study DCNNs (cf., also for brief overview of this topic see [20]).
1.2.1. Pre-DCNN Algorithms. Differences in the accuracy of face recognition algorithms as a function of race have been reported consistently since the early 1990's. One of the first studies to demonstrate a race bias in algorithms examined early neural network models based on auto-associative learning and principal components analysis (PCA) [2]. This study showed that experience-based computational models are influenced by race, but only when the basic features used to encode faces were derived from the statistical structure of the training data. The model was trained with either Asian faces, as the minority race, and Caucasian faces, as the majority race, or vice-versa. The authors found greater identification accuracy for majority-race faces than for minorityrace faces, regardless of whether Caucasian or Asian faces were the majority/minority. This suggests that identification accuracy was greater for the race with which the model had greater "experience".
A decade later, researchers examined the performance of face recognition models from the early 2000's for faces of different races [13]. These models likewise showed differential accuracy as a function of the race of the face, but only when the features used to represent faces were derived from training. Models based on dynamic link architectures, which use preset (hard-coded) features from the visual image, did not show bias, whereas models based on PCA did [21].
Race bias for algorithms has been studied also using race as a covariate [14], [15]. In 2004, Givens and colleagues found that covariates, such as race, differentially affected algorithm accuracy [14]. Covariates were measured for three algorithms [22], [23], [24]. Non-Caucasian subjects were easier to recognize than Caucasian subjects. Similarly, in a subsequent study, the three best algorithms from a 2004-2006 algorithm competition [25] showed that non-Caucasian faces (with the exception of Black faces) were recognized more accurately than Caucasian faces. Comparable results were found using a fused algorithm of the top three performing algorithms in a 2006 algorithm competition [15]. Here, non-Caucasian faces (mostly Asian) were identified more accurately than Caucasian faces. However, the effect of race was smaller than the effect of other factors, such as face size and environment (indoor or outdoor setting). Taken together, these studies show that race, as covariate, impacts algorithms in ways that are not easy to predict.
In 2011, Phillips and colleagues further explored the effects of race and found that the origin of the algorithm (i.e., the part of the world where it was developed) mediates race bias in algorithms [16]. They compared two fused algorithms: A Western algorithm (created from a fusion of 8 Western algorithms) and an East Asian algorithm (created from a fusion of 5 East Asian algorithms). Both algorithms demonstrated an other-race effect for face identification. The Western algorithm performed more accurately for Caucasian faces and the East Asian algorithm performed more accurately for East Asian faces. This is the only study, to date, that shows that algorithm origin can predict race bias. The algorithms tested in this study were "black-box" algorithms (the algorithms and their implementations were unknown to the researchers who tested for race bias). The authors speculated that differences in the racial composition of the training datasets for Western and East Asian algorithms may have contributed to the race bias.
Perhaps the best known and most comprehensive comparison done on pre-DCNN algorithms was reported in 2012 [17]. In that study, Klare et al., examined the role of demographics on the accuracy of three commercial offthe-shelf algorithms (COTS), in addition to three in-house algorithms (two non-trainable and one trainable algorithm). The researchers looked at race (Black, White, and Hispanic), as well as gender (male/female) and age (young: 18 to 30, middle-aged: 30 to 50, and old: 50 to 70). The results converged on the finding that performance for young, black, and female faces suffered relative to other demographic groups across all algorithms. Additionally, the authors found that equitable training across all groups (for the trainable algorithms) reduced the effects of specific demographics bias, but did not eliminate them.

DCNN Algorithms.
In 2014, the accuracy and generalizability of face recognition algorithms increased markedly with the introduction of algorithms based on DCNNs. These networks involve a series of pooling and convolutions of multiple layers of simulated neurons. The result of the computations is a compressed representation of a face at the top-layer of network [26]. This face descriptor can be examined directly to evaluate the quality of face codes as a function of race and other demographic variables.
Given the prominence of automatic face recognition in social media and security, it is important to know whether these new algorithms show race biases similar to those seen in previous-generation algorithms. To date, only two studies have examined race bias in DCNNs trained for face recognition [18], [19]. In 2016, El Khiyari and colleagues evaluated algorithm accuracy using single-and multi-class demographic groups including: gender (male/female), age (young, middle age, older adult), and race (Caucasian/Black). Two algorithms were tested: a COTS face recognition algorithm and a publicly available DCNN (VGG-Face algorithm [27]) [18]. In the single-class demographics group, accuracy for both algorithms was lower for female, black, and young groups. These results replicated previous research on older generation algorithms [17]. Notably, although VGG-Face had a greater overall verification accuracy, it also performed more accurately for Caucasian faces than for Black faces. Moreover, it showed more race bias than the COTS algorithm. For multi-class demographic group comparisons, accuracy varied widely. Over all comparisons, however, accuracy was greatest for middle-aged white males and was lowest for young black females.
In the most recent test available [19], Krishnapriya et al. compared accuracy for African American and Caucasian faces across four algorithms: two COTS algorithms, VGG-Face (a DCNN), and a ResNet-based DCNN [28]. The newer of the two COTS and the ResNet algorithm performed best on African American faces. By contrast, VGG-Face and the older COTS performed better on Caucasian faces. Further evidence of race bias was found in the thresholds functions for false accept rates (FARs), which differed for African American faces and Caucasian faces. (Threshold functions will be discussed in the next section).
The effect of photo quality on estimates of race bias was also examined [19]. Results showed that overall accuracy improved when only International Civil Aviation Organization (ICAO)-complaint images were used in the analysis. These results provide additional evidence of race bias in newer DCNNs, as well as a first look at how image quality can affect accuracy performance.
Lesson 2. Nearly all face recognition algorithms tested in the past 30 years show performances differences as a function of the race of the faces tested. In some cases, algorithms have shown a "human-like" interaction between the geographic origin of the algorithm and face race.

Measuring Face Identification Accuracy
Before we consider the measurement of race bias, we digress briefly to discuss the standard procedures used for measuring the accuracy of face recognition algorithms, in general. As we will see, standard methods for measuring algorithm accuracy partly underlie the difficulties we have in estimating race bias in these systems. See [29] for further discussion on this topic.
The standard approach to measuring the accuracy of face recognition algorithms is based on Signal Detection Theory (SDT) [30]. The problem is formulated as a face verification task whereby pairs of images, either of the same person or of two different people, are compared. The algorithm generates a similarity score that serves to index the likelihood that the images show the same person (high similarity) or two different people (low similarity). Accuracy is measured based on the degree of overlap between the similarity score distributions for different-identity pairs and for same-identity pairs (See Figure 1). Algorithm accuracy is summarized by the receiving operating characteristic (ROC) curves and synopsized as the area under this curve (AUC). A lower AUC score indicates a larger overlap between the two distributions, which suggests poorer discriminability. A higher AUC score indicates a less overlap between the two distributions, and greater discriminability. AUC = 0.5 indicates chance performance. AUC = 1.0 indicates perfect accuracy. ROC and AUC scores provide robust measures of overall algorithm accuracy over the entire population of face image pairs in the distributions.
In applications, identification decisions require a threshold similarity score, over which an image pair is judged as an identity "match". Once a threshold is set, additional measures of identification accuracy become relevant. These measures include false rejection rate (FRR) (proportion of same-identity pairs that have been misjudged as differentidentity pairs) and false accept rate (FAR) (proportion of different-identity pairs that have been misjudged as sameidentity pairs). Threshold placement determines algorithm accuracy at specific FRR and FAR (see Fig. 1).
A common practice is to set a threshold that yields a very small proportion of false accepts (e.g., 1/1,000, 1/10,000 or less). In an application, the critical measure of accuracy then becomes the verification rate (VR) (same-identity pairs correctly judged as same-identity pairs) at the user-set threshold. A threshold function shows FAR as a function of similarity score. As noted, threshold functions may differ by face race. We will discuss this issue in greater detail in the next section.
Lesson 3. Algorithm accuracy measures can summarize the overall performance of a system (ROC and AUC) based on the underlying distributions. Or, they can summarize performance given both a user-set threshold and the underlying distributions. Consequently, identification can be measured with threshold-independent (ROC, AUC) or threshold-dependent measures (e.g., VR @ FA=0.001). Both types of measures serve a useful function, but they are not guaranteed to converge. As we shall see in Section 2, it is easy to understand how algorithms, which appear to perform at ceiling for two races of faces using an AUC (threshold-independent) measure, actually show small, but potentially meaningful, differences in VR at the very low FARs (threshold-dependent) required for scenarios. Therefore, AUC, as a summary measure of the full range of potential thresholds, can sometimes be a misleading indication of race bias. This is especially problematic when a machine is required to operate at extreme points in the ROC function. That said, at these extreme operating points performance differences may apply to a small number of cases (i.e., stimuli in the tails of the distribution).

Factors underlying race bias
The underlying elements of race bias in algorithm performance are easily seen once we understand how the problem of face identification is formulated. The issues can be divided into data-driven and scenario-modeling issues.
1.4.1. Data-driven problems. It is clear that the race bias in algorithm performance stems from the underlying sameand different-identity distributions. The dilemma is that the distribution parameters (e.g., means, standard deviations, skews) that characterize the population as a whole, may differ for particular demographic subgroups (race, age, gender).
Differences among the subgroup distributions explain both everything and nothing about the underlying problem of bias. What we would like to know is why these distributions differ. Obvious possibilities include the following. First, it is possible that an algorithm produces representations that differ in quality (or uniqueness) for faces in different subgroups. Characteristics of the algorithm, such as the architecture or training data, may inadvertently produce biased representations. An algorithm with poor quality representations for particular subgroups would show race biases with any dataset. When the quality of the representation depends on adequate training with representative faces of a subset, we should expect bias. The racial composition of training datasets has been shown to effect the accuracy of previous generation algorithms (e.g., [2], [17]).
The problem for DCNNs may be more challenging, because they must be trained with enormous numbers of real-world (unconstrained) images. These images vary, not only in demographics, but also in quality and image factors (illumination, viewpoint, etc.). Therefore, it is difficult to isolate the impact of the racial composition of the training set on race bias. Because of the complexities of assessing the impact of training sets on DCNNs, there are currently no relevant results for these algorithms in the literature. Here, we concentrate on bias in datasets used to test DCNNs. These affect estimates of performance across race and race bias measurements.
Second, it is possible that subgroups are not equally represented in the test (i.e., measurement) distributions. For example, inaccurately represented sub-populations may introduce inherent bias into algorithms.
Third, it is possible, that the quality of the training or test photographs differs across subgroups. Krishnapriya et al. [19] found that image quality differences for the test faces explained some, but not all, of the variation in race bias. When only ICAO-compliant images were used to measure race bias, African American and Caucasian face accuracy improved across all four algorithms. The data on which race bias measurements are computed can contribute to variable estimates of race bias in face recognition algorithms.
Fourth, it is possible that nested subgroups of faces have characteristics that amplify or hide bias effects. For example, race bias effects may be stronger for female faces than for male faces; or race bias may occur for challenging stimuli, but not for easier stimuli. We will see an example of this latter case in the experiment we present.
Lesson 4. Data-driven sources of race bias are defined objectively by differences in the underlying distributions of demographic sub-populations of faces. There are multiple reasons why these differences might occur. The underlying factors are not mutually exclusive. Therefore, in any given scenario, one or more data-driven anomalies might contribute to system bias.
1.4.2. Scenario-modeling. These are directly under the control of the "user" and include: (a) thresholds effects and (b) the formulation and control of the demographic homogeneity of the different-identity distribution. Beginning with thresholds, it is clear that a uniform threshold is not adequate or equitable when the underlying sub-population distributions differ. This is a concern with race bias. Previous studies have shown that the threshold needed to achieve a consistent FAR and/or VR varies across racial groups [19], [31]. These differences underscore the importance of threshold shifts. Setting a single threshold for all racial groups may produce different estimates of FAR and VRs for different sub-groups of faces. This can be a source of race bias in the operation of algorithms.
For example, identification accuracy for Asian and Caucasian faces was measured for two different stimulus sets using ROC curves and threshold functions [31]. ROC curves showed a slight advantage for Asian faces on the easier stimulus set, and an advantage for Caucasian faces on the more difficult stimulus set. However, the threshold functions, which show the relationship between similarity score and FAR, differed for the two datasets. For both datasets, there was a greater threshold shift for Asian faces than for Caucasian faces. This threshold difference indicates that achieving equitable FARs across both racial groups, would require setting a larger threshold for Asian faces compared to Caucasian faces.
Threshold differences have been demonstrated also for African American and Caucasian faces [19]. As noted, ROC curves showed that two out of four algorithms were less race biased for African American faces compared to Caucasian faces. However, despite this advantage, threshold shifts for African American faces differed uniformly for all four algorithms. For African American faces, all four algorithms needed a higher threshold to achieve a given FAR, and a lower threshold to achieve a given VR. These results provide additional evidence that uniform thresholds across race groups may produce different accuracy results for each race group. This study [19] also provides evidence that ROC curves can obscure information that threshold functions reveal. See Lesson 3.
A second scenario-modeling concern involves the demographic homogeneity of the different-identity distribution. Common practice is to measure algorithm accuracy with all possible different-identity pairs as the baseline. However, this practice can be problematic when different-identity pairs also differ demographically (e.g., by race, gender, age). If this is the case, the distribution for different-identity pairs will include demographic differences, in addition to identity differences. Clearly, the average similarity score computed for different-identity pairs that also differ in demographic group will be lower than the average similarity score for different-identity pairs within a homogeneous demographic. This demographic heterogeneity can shift the position of the different-identity distribution leftward, thereby artificially inflating algorithm verification performance. A solution to this problem is "yoking". Yoking is defined as controlling the different-identity distribution so that all different-identity pairs are demographically comparable (e.g., same race, age, gender) [31]. This constraint produces a more homogeneous different-identity distribution (compared to including all possible different-identities in the analysis) and provides a more accurate assessment of accuracy and race bias.
As predicted, yoking alters estimates of algorithm accuracy [31], [32]. In [31], the effect of yoking on identification performance measures was demonstrated clearly. Differentidentity distributions were demographically controlled in four groups: no control, race only, gender only, race and gender. Overall accuracy decreased with increased demographic control.
Lesson 5. Scenario-modeling problems are (partially) under the control of the researcher and can directly impact estimates of race bias. System-users should consider how thresholds and yoking controls affect each race of interest independently.

Race Bias in Face Identification Algorithms
In this section, we introduce an experiment on race bias in face identification algorithms as proof of principle concerning these issues and the lessons learned. We present data on four face recognition algorithms (one pre-DCNN, and three DCNN) tested on Caucasian and Asian face pairs to evaluate race bias. Additionally, we demonstrate how yoking across race, and race and gender, affects estimates of identification accuracy across older and newer generations of DCNNs. Finally, we focused on evaluating race bias as a function of stimulus pair difficulty (see Stimuli).

Methods
2.1.1. Stimuli. Images were taken from a condensed set of the Good, Bad, Ugly (GBU) Challenge dataset [33]. The partitions of the GBU dataset provide us with the opportunity to compare race bias across algorithms and item (image pairs) difficulty level. The GBU dataset contains images partitioned into three difficulty levels called: the Good, the Bad, and the Ugly. These difficulty partitions are based on the similarity scores of the top three performers in the Face Recognition Vendor Test (FRVT) 2006 [34]. The full GBU set contains 1,085 images of 437 identities in each partition. The dataset includes frontal images that vary in environment (e.g., indoors and outdoors) and demographics (i.e., race, age, gender). The stimuli were captured using a Nikon D70 6-megapixel single-lens reflex camera. 1 All images were collected within a single academic year at the University of Notre Dame [see [33] for full details on how the GBU was compiled]. For the purposes of this study, only a subset of the full GBU dataset was selected. First, we restricted our analysis to images taken indoors, which limited the variation in illumination as a confounding factor. Second, given that Asian and Caucasian faces represented the majority of images, only these two racial groups were selected for comparison. Although this reduced the race groups we were able to test, we chose the GBU dataset for testing race bias because (1) race in the GBU dataset is self-reported, (2) the conditions under which the images were taken are identical across and within race groups, and (3) age range is narrow which limits other confounding demographics. For this reason, our analysis was performed on a condensed version of the GBU dataset (good: 385 identities; bad: 389 identities; ugly: 380 identities) (see Fig.  2 for a stimulus example).

Algorithms.
Four algorithms were used to compare verification accuracy for Asian and Caucasian faces: A2011 [33], A2015 [27], A2019 [35], A2017b [36]. None of the algorithms were trained on face images collected at Notre Dame 2 . The A2011 algorithm is a fused algorithm of three top performing algorithms in the FRVT 2006 conducted by the National Institute of Standards and Technology (NIST). This fused algorithm pre-dates DCNNs, and thus A2011 is the oldest algorithm we tested. This algorithm was selected 1. The identification of any commercial product or trade name does not imply endorsement or recommendation by NIST.
2. The training datasets for A2011 are not known. For the DCNN algorithms the training datasets are described in the original papers. as a race-bias assessment of a pre-DCNN face recognition algorithms. Moreover, A2011 has been widely used in other comparisons [37], [38]. A2015 is a publicly available DCNN and is commonly as a benchmark for performance of DC-NNs. A2015 is an older, but well established, DCNN that produces an output of 4,096-dimensional feature vectors. Previous studies have shown a race bias (greater verification accuracy for Caucasian faces compared to Black faces) for A2015 [18], [19]. The A2015 was selected because of its prominence in the literature as a baseline measure of algorithm accuracy [18], [19], [37], [39]. A2017b, which produces 512-dimensional feature vector, is based on a ResNet-101 architecture and is trained on about 5.7 million images. Most recently, A2017b was found to be comparable in accuracy to that of forensic facial examiners [39]. Finally, A2019 is based on an Inception architecture that produces 512-dimensional feature vectors also, and is trained on close to one million images. To our knowledge, no study has previously examined how A2019 and A2017b perform on different racial groups.Both A2017b and A2019 were selected because they represent state-of-the-art algorithms with welldefined training data. This study provides the first direct comparison of race bias across a pre-DCNN algorithm, a previous generation DCNN, and 2 high-performing DCNN algorithms. All are published and available for scrutiny.

Overall Accuracy.
We computed accuracy by collapsing across race and GBU partitions. ROC curves and AUC scores were calculated to measure face verification performance accuracy. ROC curves for all figures are plotted on a log-scale function to show performance at commonly set FAR = 0.0001 and 0.001 (gray dotted lines) and are race and gender yoked. Overall accuracy was best for the two newer algorithms, A2019 and A2017b, followed by A2011, and A2015 (Fig. 3).  Figure 3. ROC curves for A2019 (yellow), A2017b (green), A2015 (blue), A2011 (red). A2019 and A2017b show near perfect performance, followed by A2011, and A2015.

2.2.2.
Yoking. The effects of yoking are easily seen on overall accuracy by comparing results with three yoking conditions. For the no yoking condition, all available differentidentity pairs were considered, regardless of cross-race and cross-gender status of identities. For the race-yoking condition, only same-race different-identity pairs were considered. For the race and gender-yoking condition, only same-race and same-gender different-identity pairs were considered. Fig. 4 shows that algorithms showed yoking effects in the predicted directions (accuracy decreased as yoking constrains increased), but that the magnitude of the effects varied. The effect of yoking on overall accuracy was most evident for both A2011 and A2015, and less so for A2017b and A2019. (See Lesson 5).  Table 1. As can be seen, by the AUC measure, performance is near ceiling (AUC = 1.0) in all cases. By this account, all four algorithms show little to no race bias on overall accuracy for Caucasian and Asian faces. Examination of the verification estimates at low FARs, however, tell a different story (Fig. 5). These reveal effects of race bias for all algorithms. For A2019 and A2017b, there was no or little bias for FARs larger than 0.001; however, for smaller FARs there was bias.
These results demonstrate a case where overall accuracy (threshold-independent) and accuracy at a pre-determined threshold (threshold-dependent) may lead to different conclusions, despite the fact they are internally consistent (See Lesson 2 and Lesson 3).   2.3.2. Thresholds. Next, we calculated FAR as a function of the different-identity similarity-scores for Asian and Caucasian faces (see Fig. 6). For all four algorithms, the Asian threshold function (orange) is always to the right of the Caucasian function (teal) 3 . Because of this shift, a fixed threshold will yield a smaller FAR for Caucasian faces than for Asian faces. To obtain the same FAR for both races will 3. Because the similarity score scales differ across algorithms, direct comparisons for threshold shift magnitudes across algorithms is not possible.

Item Difficulty
Performance on each GBU partition was calculated (Fig.  7). A2019 and A2017b were more accurate than A2015 and A2011 across all three partitions. A2015 had the lowest verification accuracy for the Good and Bad images, whereas A2011 had the lowest accuracy for Ugly. These data are comparable to previous research that shows the trade-off between A2015 and A2011 in the Bad and Ugly partitions [37]. Accuracy for the Ugly partition was the lowest across all three difficulty groups.
2.4.1. Race accuracy as a function of Item Difficulty. The GBU dataset allows us to examine race bias as a function of item difficulty. We computed ROC curves for Asian and Caucasian faces across the three difficulty levels of GBU (Fig. 8). Accuracy for the Good partition is nearly perfect for both Asian and Caucasian faces. A2015, and to a lesser extent A2011, showed a race bias in favor of Caucasian faces at FAR = 0.0001. For the Bad partition, A2017b and A2019 again showed nearly no race bias, whereas A2011 and A2015 showed greater verification accuracy Caucasian faces at FAR = 0.0001. For the Ugly partition, no algorithm achieved perfect performance for either race. All algorithms, except for A2011, showed greater accuracy for Caucasian faces compared to Asian faces. (See Lesson 4).

Discussion
We reviewed and analyzed relevant literature on race bias in face recognition from humans and algorithms published over the last 50 years. Five lessons in measuring race bias for algorithms emerge from this review of past work. We illustrated the lessons' effects on measuring bias in a series of experiments with three recent DCNN algorithms and one pre-DCNN algorithm. The factors that affect bias fall into two categories: data-driven and scenario-based. The sources of bias that a researcher must consider include the following: Each of these factors individually, as well as interactions among factors, can impact bias. Data-driven factors underlie real differences in an algorithm's capacity to recognize faces of different races. Scenario-based factors are part of the measurement process and affect our estimates of algorithm bias. Given the complexity of these factors, and their potential to interact, it is not possible to perform a general assessment of bias for face recognition algorithms. Consequently, race bias must be measured for each particular scenario, algorithm, race, and dataset. The five lessons we present provide a starting point for developing a principled understanding of how race and demographic bias should be assessed for face recognition algorithms.
One obvious limitation in our study was the use of only two racial groups in our analysis. As noted, the GBU dataset was selected because of the excellent control it provides of factors other than race. This photometric and demographic control makes it ideal for studying race bias, but somewhat limits the ecological validity. Although the analysis was limited to two groups, the lessons learned and methodological considerations apply across all race groups. Moreover, the focus on only two races allowed us to explore a wide range of possible factors that may contribute to bias.
Race bias in face recognition algorithms is a critical problem that remains unsolved. Although there are promising approaches on the horizon (cf., [40]), these methods need to be tested more thoroughly. In many cases, they are still constrained to work on specific challenges inherent in particular scenarios that may, or may not, generalize to other types of problems. The holistic assessment provided here is integral to understanding that the solution to the race bias problem is not straightforward. The intricacy of this issue is underscored by the multitude of underlying factors that impact race bias.
With the rise of DCNNs, the complexity of the bias problem increases. This is due to the need for extremely large training sets and the large number of parameters that may impact performance on subsets of faces. Until a technical solution to the problem of bias is found, algorithms need to be tested individually and thoroughly for performance across racial groups.

Conclusion
This is the first study to consider the full range of factors that impact race bias in face recognition algorithms. Attention to race bias factors in past work has often been piecemeal. For example, common focus has been on the role of training data, and although this is an important factor, there remains a broad scope of other factors to consider. At this point in time, advances in the field require simultaneous attention to all potential sources of race bias in face recognition algorithms. Both data-driven and scenario-modeling factors and their interactions can impact race bias in face recognition algorithms. This holistic assessment provides a realistic and informed starting point for future studies in this area.