{'fraction_non_alphanumeric': 0.03359201172192978, 'fraction_numerical': 0.017799967439083953, 'mean_word_length': 5.124127617148554, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
AFR-Net: Attention-Driven Fingerprint Recognition Network

The use of vision transformers (ViT) in computer vision is increasing due to limited inductive biases (e.g., locality, weight sharing, etc.) and increased scalability compared to other deep learning methods. This has led to some initial studies on the use of ViT for biometric recognition, including fingerprint recognition. In this work, we improve on these initial studies for transformers in fingerprint recognition by i.) evaluating additional attention-based architectures, ii.) scaling to larger and more diverse training and evaluation datasets, and iii.) combining the complimentary representations of attention-based and CNN-based embeddings for improved state-of-the-art (SOTA) fingerprint recognition (both authentication and identification). Our combined architecture, AFR-Net (Attention-Driven Fingerprint Recognition Network), outperforms several baseline transformer and CNN-based models, including a SOTA commercial fingerprint system, Verifinger v12.3, across intra-sensor, cross-sensor, and latent to rolled fingerprint matching datasets. Additionally, we propose a realignment strategy using local embeddings extracted from intermediate feature maps within the networks to refine the global embeddings in low certainty situations, which boosts the overall recognition accuracy significantly across each of the models. This realignment strategy requires no additional training and can be applied as a wrapper to any existing deep learning network (including attention-based, CNN-based, or both) to boost its performance.


INTRODUCTION
A UTOMATED fingerprint recognition systems have continued to permeate many facets of everyday life, appearing in many civilian and governmental applications over the last several decades [1]. Due to the impressive accuracy of fingerprint recognition algorithms (0.626% False Non-Match Rate at a False Match Rate of 0.01% on the FVCongoing 1:1 hard benchmark [2]), researchers have turned their attention to addressing difficult edge-cases where accurate recognition remains challenging, such as partial overlap between two candidate fingerprint images and crosssensor interoperability (e.g., optical to capacitive, contact to contactless, latent to rolled fingerprints, etc.), as well as other practical problems like template encryption, privacy concerns, and matching latency for large-scale (gallery sizes on the order of tens or hundreds of millions) identification.
For many reasons, some of which mentioned above (e.g., template encryption and latency), methods for extracting fixed-length fingerprint embeddings using various deep learning approaches have been proposed. Some of these methods were proposed for specific fingerprint-related tasks, such as minutiae extraction [3], [4] and fingerprint indexing [5], [6], whereas others were aimed at extracting a single "global" embedding [7], [8], [9]. Of these methods, the most common architecture employed is the convolutional neural network (CNN), often utilizing domain knowledge (e.g., minutiae [8]) and other tricks (e.g., specific loss functions, such as triplet loss [10]) to improve fingerprint recognition accuracy. More recently, motivated by the success of attention-based Transformers [11] in natural language processing, the computer vision field has seen an influx of the use of the vision transformer (ViT) architecture for various computer vision tasks [12], [13], [14], [15].
In fact, two studies have already explored the use of ViT for learning discriminative fingerprint embeddings [16], [17]; albeit, with the following limitations: i.) the authors of [16] supervised their ViT model using a pretrained CNN as a teacher model and thus did not give the transformer architecture the freedom to learn its own representation and ii.) the authors of [17] were limited in the data and choice of loss function used to supervise their transformer model, thereby limiting the fingerprint recognition accuracy compared to the baseline ResNet50 model. Nonetheless, the authors in [17] did note the complimentary nature between the features learned by the CNN-based ResNet50 model and arXiv:2211.13897v2 [cs.CV] 3 Dec 2022 the attention-based ViT model. This motivated us to evaluate additional attention-based models that bridge the gap between purely CNN and purely attention-based models, in order to leverage the benefits of each. Toward this end, we evaluate two ViT variants (vanilla ViT [12] and Swin [15]) along with two variants of a CNN model [18] (ResNet50 and ResNet101) for fingerprint recognition. In addition, we propose our own architecture, AFR-Net (Attention-Driven Fingerprint Recognition Network), consisting of a shared feature extraction and parallel CNN and attention classification layers.
Even though these models are trained to extract a single, global embedding representing the identity of a given fingerprint image, we make the observation that for both CNNbased and attention-based models, the intermediate feature maps encode local features that are also useful for relating two candidate fingerprint images. Correspondence between these local features can be used to guide the network in placing attention on overlapping regions of the images in order to make a more accurate determination of whether the images are from the same finger. Additionally, these local features are useful in explaining the similarity between two candidate images by directly visualizing the corresponding keypoints, as shown in Figure 1.
More concisely, the contributions of this research are as follows: • Analysis of various attention-based architectures for fingerprint recognition.
• Novel architecture for fingerprint recognition, AFR-Net, which incorporates attention layers into the ResNet architecture.
• State-of-the-art (SOTA) fingerprint recognition performance (authentication and identification) across several diverse benchmark datasets, including intrasensor, cross-sensor, contact to contactless, and latent to rolled fingerprint matching.
• Novel use of local embeddings extracted from intermediate feature maps to both improve the recognition accuracy and explainability of the model.

•
Ablation analysis demonstrating the importance of each aspect of our model, including choice of loss function, training dataset size, use of spatial alignment module, and use of local embeddings to refine the global embeddings.

RELATED WORK
Here we briefly discuss the prior literature in deep learningbased fingerprint recognition and the use of vision transformer models for computer vision. For a more in-depth discussion on these topics, refer to one of the many survey papers available (e.g., [19] for deep learning in biometrics and [20] for the use of transformers in vision).
Among the first studies on extracting global fingerprint embeddings using deep learning was proposed by Li et al. [7], which used a fully convolutional neural network to produce a final embedding of 256 dimensions. The authors of [8] then showed improved performance of their fixed-length embedding network by incorporating minutiae domain knowledge as an additional supervision. Similarly, Lin and Kumar incorporated additional fingerprint domain knowledge (minutiae and core point regions) into a multi-Siamese CNN for contact to contactless fingerprint matching [9]. More recently, [16] and [17] proposed the use of vision transformer architecture for extracting discriminative fixed-length fingerprint embeddings, both showing that incorporating minutiae domain knowledge into ViT improved the performance.

Vision Transformers for Biometric Recognition
Transformers have led to numerous applications across the computer vision field in the past couple of years since they were first introduced for computer vision applications by Doesovitskiy et al. in 2021 [12]. The general principle of transformers for computer vision is the use of the attention mechanism for aggregating sets of features across the entire image or within local neighborhoods of the image. Attention was originally introduced in 2015 for sequence modeling by Bahdanau et al. [29] and has been shown to be a useful mechanism in general for operations on a set of features. Today, numerous variants of ViT have been proposed for a wide range of computer vision tasks, including image recognition, generative modeling, multi-model tasks, video processing, low-level vision, etc. [20].
Some recent works have explored the use of transformers for biometric recognition across several modalities including face [30], finger vein [31], fingerprint [16], [17], ear [32], gait [33], and keystroke recognition [34]. In this work, we improve upon these previous uses of transformers for fingerprint recognition by evaluating additional attention-based architectures for extracting global fingerprint embeddings.

AFR-NET: ATTENTION-DRIVEN FINGERPRINT RECOGNITION MODEL
Our approach consists of i.) investigating several baseline CNN and attention-based models for fingerprint recognition, ii.) fusing a CNN-based architecture with attention into a single model to leverage the complimentary representations of each, iii.) a strategy to use intermediate local feature maps to refine global embeddings and reduce uncertainty in challenging pairwise fingerprint comparisons, and iv.) use of spatial alignment module to improve recognition performance. Details of each component of our approach are given in the following sections.

Baseline Methods
First, we improve on the initial studies ( [16], [17]) applying ViT to fingerprint recognition to better establish a fair Fig. 2: Overview of the AFR-Net architecture. First, input fingerprint images are passed through a spatial alignment module for better alignment of two fingerprints under comparison, then passed through a shared feature extraction, followed by two classification heads (one CNN-based and the other attention-based). For our implementation, we followed the ResNet50 architecture as our backbone and CNN classification head and used 12 multi-headed attention transformer encoder blocks for the attention-based classification head.
baseline performance of ViT compared to the CNN-based models. This is accomplished by removing the limitations of the previous studies in terms of choice of supervision and size of training dataset used to learn the parameters of the models. We then compare ViT with two variants of the ResNet CNN-based architecture, ResNet50 and ResNet101. For our specific choice of ViT, we decided on the small version with patch size of 16, number of attention heads of 6, and layer depth of 12. We selected this architecture as it presents an adequate trade-off in speed and accuracy compared to other ViT architecture variants. In addition, we compare the performance of a popular ViT successor, Swin, which utilizes a hierarchical structure and shifted windows for computing attention within local regions of the image. Specifically, we used the small Swin architecture with patch size of 4, window size of 7, and embedding dimension of 96. For another strong baseline comparison, we included the latest version of the SOTA fingerprint recognition system from Neurotechnology, Verifinger v12.3, for our evaluations 1 . According to the FVC On-going competition, Verifinger is the top performing algorithm for the 1:1 fingerprint verification benchmark [2].

Proposed AFR-Net Architecture
Based on previous research suggesting the complimentary nature of ViT and ResNet embeddings, we were motivated to merge the two into a single architecture, referred to as AFR-Net. As shown in Figure 2, AFR-Net consists of a spatial alignment module, shared CNN feature encoder, CNN classification head, and an attention classification head. The shared alignment module and feature encoder greatly reduces the number of parameters compared to the fusion of the two separate networks and also allows 1. https://neurotechnology.com/verifinger.html the two classification heads to be trained jointly. The full architectural details of AFR-Net are given in Table 1.
Due to the two classification heads, we have two bottleneck classification layers which map each of the 384-d embeddings, Z c and Z a , into a softmax output representing the probability of a sample belonging to one of N classes (identities) in our training dataset. We employ the Additive Angular Margin (ArcFace) loss function to encourage intra-class compactness and inter-class discrepancy of the embeddings of each branch [35]. Through an ablation study, presented in section 4.5, we find that despite the relatively little use of this loss function in previous fingerprint recognition papers [17], [36], the ArcFace loss function makes an enormous difference in the performance of our model.

Global Embedding Refinement via Local Embeddings
As noted in the introduction, and demonstrated in Figure 1, we find that the intermediate feature maps of our AFR-Net model (and in general, all the deep learning models evaluated in this work) encode local descriptors (i.e., embeddings) of the input images. These local descriptors can be matched between two fingerprint images and used to compute a correspondence between similar regions. Given the surprising accuracy of these local embeddings in locating corresponding points of interest between two images, we devise a strategy to use these corresponding regions of interest as a sort of hard attention for the model to refine the global embeddings based on just the overlapping regions present in both images.
Some examples of this process are demonstrated in Figure 3, where the correspondence between local embeddings is used to compute an affine transformation between the image pairs. Then, the non-overlapping fingerprint regions are masked and each image is presented to the network for a second time to yield a new set of embeddings. Finally, a second similarity score between the masked images is computed via a cosine similarity between the new embeddings. The similarity between the masked regions is combined via a weighted sum with the similarity score obtained from the original images to obtain a final similarity score. For ResNet50, ResNet101, and AFR-Net, we take the last output of the Conv4 layer as our local embeddings, which has dimensions of 14x14x1024. For ViT and Swin, we take the final patch embeddings at the output of the last attention layer as the local embeddings, which has dimensions of 14x14x384. In all cases, each of these 196 local descriptors corresponds to a single 16x16 patch of the input fingerprint image. We assign the center of each patch as the keypoint associated with the corresponding local embedding when computing the correspondence points between two finger- Indeed, computing the correspondence between sets of local descriptors of two images is time consuming, especially if computing a brute force exhaustive search to establish a 1:1 correspondence between matched descriptors. For this reason, we only employ the re-weighting strategy in low certainty scenarios (when the similarity score is close to the match threshold) to keep the amortized latency of our algorithm approximately the same as without the re-weighting process; that is, we only utilize the local descriptors if the similarity score between the original global embeddings falls between a specified range [s l , s h ]. Values of 0.3 and 0.6 for s l and s h , respectively, were empirically determined on our validation dataset to work well across all our models. Furthermore, if a valid homography computed between corresponding local regions cannot be obtained (e.g., if the scale, rotation, and/or translation parameters exceed expected limits), we fall back to the original similarity score as to not further degrade the comparison by computing a new set of embeddings from images which have been corrupted due to poorly behaved transformation matrices. Figure 4, shows the genuine and imposter score distributions for our AFR-Net model on the FVC 2002 DB3A dataset, where we experienced the biggest increase in performance after reweighting the predictions using this method. In figure 4, we show (a) the original score distributions, (b) the scores computed on the refined embeddings, and (c) the fused score distributions after the weighted averaging. The full algorithm of this process is detailed in Algorithm 1.

15:
M ← getHomography(kp1, kp2) 16: if homographyOK(M ) then 17: 19: end if 24: end if 25: 26: return s 27: end procedure There are likely more sophisticated, faster alternatives to a brute force algorithm for computing the correspondence between sets of local embeddings, as well as better ways of aggregating the similarity scores between matched local embeddings themselves. However, we leave those areas of exploration for future work as the current algorithm seems to improve the results significantly across all models and datasets and has the nice interpretation of giving the network a second glance at regions of interest in uncertain cases -much like a human fingerprint examiner would. Nonetheless, some suggestions for future extensions would include the use of a graphical neural network (GNN) or attention mechanism to more intelligently aggregate the sets of local descriptors between two images.

Spatial Alignment Module
As has been noted in previous literature on fingerprint recognition [8], [37], [38], [39], [40], Spatial Transformer Networks [41] have been shown to be highly effective in aligning input fingerprints for improved recognition accuracy across a wide range of tasks (e.g., contact to contactless fingerprint matching, partial fingerprint recognition, etc.). That, coupled with our observation that the local descriptors used in our realignment procedure are not rotation invariant, we were motivated to include a spatial alignment mod-ule into the architecture of our AFR-Net model and each of the baseline models. The details of our spatial alignment module are given in Table 1, along with the details of the rest of the AFR-Net network architecture. Lastly, in the ablation portion of our experimental section, we further emphasize the benefit of incorporating the spatial alignment module into our fingerprint recognition network.

Training Details
AFR-Net and all baseline models, excluding Verifinger, were trained with an ArcFace loss function with a margin of 0.5, learning rate of 1e-4, weight decay of 2e-5, and polynomial learning rate decay function with a power of 3 and minimum learning rate of 1e-5. The AFR-Net, ResNet100, and Swin models were trained with a batch size of 64 across four Nvidia GeForce RTX 2080 Ti GPUs, whereas the ResNet50 and ViT models were trained with a batch size of 128. AFR-Net, ResNet50, and ResNet100 were trained with the Adam optimizer [42] and ViT and Swin were trained with the AdamW optimizer [43]. The maximum number of epochs for all models was set to 75; however, the number of epochs trained for the final saved models varied based on the highest validation accuracy computed during training on a hold-out validation dataset. Finally, we initialize each model using the pre-trained ImageNet weights made available by the open-sourced pytorch-image-models git repository [44].

EXPERIMENTAL RESULTS
In this section, we discuss the training and evaluation datasets used, the authentication and identification results achieved by our method in comparison with the baseline methods, latency and performance trade-off between the methods, and an ablation analysis to highlight the contributions of individual components of our algorithm.

Datasets
For training our models, we aggregate a large number of fingerprint recognition datasets with diverse characteristics ranging from rolled fingerprints [45], [46], plain (i.e., slap) fingerprints, mixture of rolled and plain fingerprints [47], contactless (e.g., from mobile phone cameras) fingerprints [48], [49], [50], latent fingerprints (from the Michigan State Police (MSP) Latent Database), and even recently released synthetic fingerprints [26], [27], [28]. Example images from each of these datasets are shown in Figure 5. A small portion of the total training dataset was reserved for validation. In total, our aggregated training dataset contains 1.3M images for training and 3,814 images for validation. Further information regarding the number of unique fingers, images per dataset, and fingerprint type is given in Table 2.
Our evaluation datasets are just as diverse as our training datasets and include challenging scenarios such as contact to contactless fingerprint matching [37], [51], varying sensor types for both rolled and slap prints (e.g., optical, capacitive, thermal swipe, etc.) [52], [53], [54], latent to rolled fingerprint matching [55], and even rolled to plain

Authentication Results
We report authentication performance of our method across 11 different evaluation datasets of varying characteristics. The results are given in Table 3 as the true accept rate (TAR) at a false accept rate (FAR) of 0.01% (FAR=0.1% in the case of the FVC datasets in order to follow the established protocols) and the full Receiver Operating Characteristic (ROC) curves are given in the appendix. Besides for the established protocol on the FVC datasets, we compute all possible genuine and imposter pairs for our evaluations.
According to the results in Table 1, AFR-Net outperforms the baseline methods on 9 out of the 11 datasets and shows competitive performance on the two datasets where it comes in second place (99.96% vs. 100% and 99.36% vs. 99.54% for FVC 2002 DB2A and DB3A, respectively). We show especially impressive performance in cross-sensor (TAR=96.11% on NIST SD 302) and contact to contactless matching (TAR=98.73% and TAR=98.70% on PolyU and ZJU datasets, respectively), as well as latent to rolled fingerprint matching on the challenging NIST SD 27 dataset, where our method out performs Verifinger v12.3 (TAR=63.18% to TAR=61.63%).
AFR-Net, and even our baseline ResNet and ViT variants, show substantial improvement over previous fixedlength, global representation networks for fingerprint recognition. For example, DeepPrint, one of the top performing models in the open literature, achieves a TAR of 97.53% and 98.55% on FVC 2004 DB1A and NIST SD 14, respectively. However, given the older architectures (Inception v4 in the case of DeepPrint), loss functions, and smaller training datasets, this increase in performance is not all that surprising. In fact, this is why we benchmarked our AFR-Net model against several more recent architectures like ResNet, ViT, and their variants; all of which were trained and evaluated on the same datasets, allowing for a more fair comparison with AFR-Net.
For all the methods, we show improved performance with using the local embeddings to realign the images as a way to refine the global embeddings and improve the resulting similarity scores. The performance improvement was most pronounced for datasets with frequent partial fingerprints, such as FVC 2002 DB3A and DB1A. For ex- Fig. 5: Example image pairs from each of the datasets used in this paper. See Table 2 for details. ample, the average performance across all the methods on FVC 2002 DB3A improved from 94.46% to 96.26%, a 32.5% reduction in error. Intuitively, this realignment process has the effect of slightly improving the similarity scores between borderline genuine fingerprint pairs, by forcing the network to focus on overlapping regions in the images, and does not appreciably effect the borderline imposter scores. Thereby, pushing some of the borderline genuine matches above the rejection threshold.
If comparing just the CNN-based models (ResNet50 and ResNet101) vs. the attention-based models (ViT and Swin), the performance in terms of matching accuracy is quite comparable; however, in terms of number of parameters, ViT and Swin have substantially smaller footprints. As a result, the training time to reach convergence of these networks was significantly faster than the ResNet models, especially for ViT due to it's low latency as well (which is comparable to ResNet50). Finally, for the most part, Swin outperformed ViT in terms of accuracy across many of the datasets, but it does have more than twice the parameters and 3 times the latency of ViT, making it perhaps not as preferable in some situations.

Identification Results
We used the NIST SD 27 latent fingerprint dataset and a gallery of 100K rolled fingerprints from the MSP fingerprint dataset to evaluate the closed-set identification (i.e., 1:N search) performance of our models 3 . According to the cumulative match characteristic (CMC) curve shown in Figure 7 and the identification performance at specific retrieval ranks given in Table 7, AFR-Net is competitive with Verifinger v12.3 and outperforms all the rest of the baseline methods by a substantial margin. The rank-1 accuracy of Verifinger is 55.04%, compared to 53.10% with AFR-Net but AFR-Net surpasses Verifinger at higher retrieval ranks. The next closest performing model was ResNet101, with a rank-1 accuracy of 44.96%. Some examples image retrievals when (a) the correct mate was returned at rank-1 and (b) when the correct mate was not returned in the top five images are shown in Figure 6. In the successful case, the latent probe image is of relatively high quality and is able to match with it's corresponding mate with a similarity score far above the other returned matches; however, in the failure case, the latent image is of very poor quality and returns high similarity scores with other poor quality images in the gallery.
Despite impressive performance of our model compared to the baseline methods, we should note that latent fingerprint identification is a challenging task that requires targeted segmentation, enhancement, and matching strategies to achieve SOTA performance, as is demonstrated in these prior latent identification studies [36], [56], [57], [58]. For our evaluation, we only used manual bounding box annotations to locate the latent fingerprints prior to matching, but we did not use any other preprocessing or enhancement; thus, our performance could be further improved for latent to rolled fingerprint matching. Additionally, since we do not use minutiae or any other fingerprint domain knowledge in 3. These 100K images are completely disjoint from the 448K fingerprint images from MSP used for training. designing AFR-Net, our model may be at a disadvantage compared to the SOTA latent matchers, since minutiae have been shown to be a useful feature for matching very low quality latents [58]. Nonetheless, AFR-Net still performs reasonably well compared to Verifinger, which is also not intended for latent to rolled fingerprint matching but we can assume does incorporate some fingerprint domain knowledge (enhancement, minutiae, etc.).
Furthermore, we observed that the fusion of the two matchers (Verifinger v12.3 plus AFR-Net) leads to a significant boost in retrieval accuracy (rank-1 accuracy of about 64% compared to 55.04% for Verifinger and 53.10% for AFR-Net). Still, there is room for improvement as the SOTA rank-1 retrieval rate for NIST SD 27 against a gallery of 100K rolled fingerprint is 65.7%, according to Cao et al. [58]. We also evaluated the fusion of ResNet50 and ViT, which performed worse compared to using just AFR-Net (rank 1 retrieval rate of 49.61% vs. 53.10%). Thus, not only does incorporating both architectures into one save on latency and model size, as is done in AFR-Net, it also leads to better fingerprint recognition performance over the fusion of both individual models.
Lastly, we evaluated our model's performance for rolled to rolled fingerprint search using NIST SD 14. Consistent with previous studies [8], we used the last 2,700 images from NIST SD 14 as probes and their corresponding mates with the same 100K rolled images from MSP as the gallery. AFR-Net achieves a rank 1 retrieval rate of 99.78%, which is an improvement over the previous SOTA performance of 99.20% by DeepPrint [8].

Latency
The inference speed of each method is given in Table 3, along with the number of parameters of each network. Of the models that we compared, the one with the least number of parameters is ViT (21.83M), followed by Swin (52.69M) and ResNet50 (62.21M). ViT also has the lowest latency of 4.12ms, followed closely by ResNet50 with a latency of 4.34ms. AFR-Net has 85.02M parameters, roughly equivalent to the number of parameters as ResNet50 and ViT combined, but is still comparable to the number of parameters as ResNet101 (81.20M).
In terms of performance vs. latency trade-off, ResNet50 outperformed ResNet101 on the majority of the evaluation datasets, whereas Swin outperformed ViT on the majority of the datasets; however, at a significant cost to latency and larger number of parameters. Thus, it seems that both ResNet50 and ViT may be preferable in some applications that require smaller footprints and faster inference speed. AFR-Net performed the best overall in terms of performance; however, does have a small added latency and increase in number of parameters compared to, for example, ResNet50. However, the significant improvements in performance on many of the datasets seem to justify the added computational costs.
Lastly, the realignment stage utilizing the local embeddings does incur some additional latency, which we will denote as t R . For our implementation, the average value of t R is 29.36ms. In addition to t R , the realignment stage includes the time required for one additional inference time of the  In the successful case, the latent probe image is matched with its corresponding mate with a similarity score of 0.98; whereas, in the unsuccessful case, the poor quality latent image produces high similarity scores with other low quality images in the gallery and is not matched with the correct mate until rank 10 (with a similarity score of 0.84). embedding network, t I . However, since we only invoke the realignment stage for a fraction of the total comparisons, r, the amortized latency cost, t A , of the realignment is significantly lower and can be computed with the following equation: For example, with a specified range of [0.3, 0.6], the realignment process for AFR-Net is invoked 17.9% (r=0.179) of the time, on average across all the datasets. Using the inference speed of AFR-Net from Table 3 of 8.42ms, the total cost of AFR-Net † (AFR-Net with realignment) is 15.18ms.

Ablation Analysis
In the abalation study of our AFR-Net model, we analyzed the effects of the loss function (cross entropy vs. ArcFace), training dataset size, use of a spatial transformer network (STN) for spatial alignment, and our realignment strategy using the local feature embeddings. For the ablation on the training dataset size, we compared the performance of In comparison, our full training database consists of the same 675K synthetic images plus an additional 540K real fingerprint images.
The results of the ablation study are given in Table 5. The largest increase in performance is attributed to the use of ArcFace loss rather than a cross entropy loss for supervision. Interestingly, training with ArcFace loss on a subset of only publicly available training data (85K real fingerprints + 675K synthetic compared to our full dataset of 540K real fingerprints + 675K synthetic) achieves competitive recognition performance across all datasets, where really the benefit of additional data is seen in only the crosssensor and latent matching scenarios. We then saw even further improvements obtained with the incorporation of the spatial alignment network. Finally, we noticed consistent performance improvements across all evaluation datasets with applying our realignment strategy, especially in the more challenging datasets such as NIST SD 302 and FVC 2002 DB3A, which have many partially overlapping fingerprints.

DISCUSSION
In this section, we discuss some remaining failure cases of our model and some possible future extensions to mitigate those. We also investigate the robustness of our model to partial fingerprints by manually generating affine and occlusion deformations (of varying magnitudes) across multiple fingerprint datasets.

Failure Case Analysis
Two example fingerprint image pairs that failed to be successfully matched by our AFR-Net model are shown in Figure 8. As demonstrated with these representative examples, the majority of the failure cases can be attributed to one of two factors: i.) extremely poor image quality or ii.) very little overlap between the images. The first cause can be avoided by implementing a quality check into the algorithm, whereas the second cause may be more difficult to detect and/or avoid in a practical system (especially one operating with a limited acquisition aperture). Our realignment strategy is effective at improving partial overlap pairs; however, when the amount of overlap is severe, such as example (b) in Figure 8, the model may still fail.

Robustness to Occlusions and Affine Transformations
To help understand the difference between CNN-based and attention-based embeddings, we conducted an experiment to visualize the saliency maps of each model on pairs of partial fingerprints. Specifically, we scan a patch of 16x16 background pixels (with a stride of 16) across one image in the pair and compute the resulting similarity scores, which we use to draw a heatmap of salient regions for each patch in the image. We repeat this process for the other image and overlay the heatmaps onto the original images. Due to space limitations, we just show one representative example in Figure 9, along with the ridge overlays of the two images to better visualize the overlapping regions. Comparing the saliency maps of ResNet50 ((a) in Figure 9) to ViT ((b) in Figure 9), it seems that occluding some areas of the fingerprint has more of an effect on varying the similarity scores for ViT than it does for ResNet50. This suggests that ViT is placing more weight on specific regions of the fingerprint, compared to ResNet50 which may be using more of the fingerprint area for its prediction. Finally, comparing both saliency maps with the saliency map of AFR-Net (shown in (c) of Figure 9), we can see that AFR-Net exhibits characteristics of both models.
We performed two additional experiments using manual occlusions and affine transformations to generate increasingly occluded and cut-off partial fingerprints and plotted the performance of each model vs. the amount of degradation. Specifically, we generated random occlusions and affine transformations at five different ratios, corresponding to the percentage of the fingerprint area being obscured. We repeated the experiment 5 times at each ratio and recorded the average performance of each model. For reference, example images with random affine transformations and occlusions at ratios of 40% and 20%, respectively, are shown in Figure 10.
According to Figure 11, the ResNet models appear to have a slight edge over the attention-based models (ViT and Swin) when subjected to random occlusions, whereas the ViT and Swin models show more robustness to severe degrees of affine transformations compared to the ResNet models. However, AFR-Net shows the best robustness to both occlusions and affine transformations, underscoring the benefit of merging the two complimentary networks.

CONCLUSION
In this paper, we evaluated attention-based fingerprint recognition networks against competitive CNN baselines and a state-of-the-art commercial fingerprint recognition system, Verifinger v12.3, and showed that our combined architecture, AFR-Net (Attention-Driven Fingerprint Recognition Network), outperforms all of the baselines in the majority of the evaluation datasets. These evaluations included intra-sensor, cross-sensor, contact to contactless, and latent fingerprint matching scenarios. Furthermore, we introduced a realignment stage using the correspondence between local embeddings extracted from intermediate feature maps of two fingerprint images which consistently improved the performance across all the models, especially in challenging cases (e.g., partial overlap between the fingerprint images). This realignment strategy requires no additional training and can be applied as a wrapper to any deep learning network (CNN or attention-based). It also serves as an explainable visualization of the corresponding regions of two fingerprint images as ascertained by the network. Future work will aim at improving the realignment strategy to reduce the latency introduced by the current brute force correspondence implementation. We will investigate the use of attention and/or graphical neural networks for this purpose in order to more intelligently aggregate two sets of local embeddings.