{'fraction_non_alphanumeric': 0.028609967343461593, 'fraction_numerical': 0.013062615362771547, 'mean_word_length': 5.283229259589652, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
MobiBits: Multimodal Mobile Biometric Database

This paper presents a novel database comprising representations of five different biometric characteristics, collected in a mobile, unconstrained or semi-constrained setting with three different mobile devices, including characteristics previously unavailable in existing datasets, namely hand images, thermal hand images, and thermal face images, all acquired with a mobile, off-the-shelf device. In addition to this collection of data we perform an extensive set of experiments providing insight on benchmark recognition performance that can be achieved with these data, carried out with existing commercial and academic biometric solutions. This is the first known to us mobile biometric database introducing samples of biometric traits such as thermal hand images and thermal face images. We hope that this contribution will make a valuable addition to the already existing databases and enable new experiments and studies in the field of mobile authentication. The MobiBits database is made publicly available to the research community at no cost for non-commercial purposes.


Introduction
Implementation of biometric authentication on mobile devices in the recent years has lead to an almost universally ubiquitous presence of biometrics in our daily lives. Starting with the fingerprint sensor of the iPhone 5s in 2013, more than 700 mobile devices have since supported this kind of authentication, with many manufacturers employing other biometric characteristics: iris in Samsung Galaxy S8 or Microsoft Lumia 950, and recently also face in 3D with Apple iPhone X's Face ID. Today, biometric authentication is used for unlocking a phone, but also for protecting financial assets in payment systems, e.g., Apple Pay. With this reliance upon mobile identity, secure and convenient authentication methods are crucial, hence the recently increasing interest in biometric authentication for mobile. Biometric traits such as face and fingerprint have already been given considerable attention in the scientific community, therefore, the main contributions of our paper include: (1) a multimodal biometric database -MobiBits, comprising representations of five different biometric characteristics, acquired with sensors embedded in commercially available, consumer-grade mobile phones, (2) a novel use of thermal imaging enabled smartphone, adding a new dimension to typical, visible spectrum images of face and hand, offering additional cues that can be utilized, e.g., for Presentation Attack Detection, (3) experiments employing commercial and academic methods, which provide benchmark results of recognition performance for the introduced datasets.
According to our knowledge, our dataset offers the largest number of biometric characteristics represented in a single database, and employs the richest collection of data acquisition scenarios, which are trying to mimic those of potential real-world applications.
2 Related work Iris. Iris recognition, usually carried out with specialized devices operating in near infrared (NIR), poses several challenges when implemented on mobile -most importantly, the need to utilize visible light images. Raja et al. investigated visible spectrum iris recognition on mobile devices by employing deep sparse filtering [Ra14] and K-means clustering [RRB14], being able to achieve equal error rate (EER) of 0.31%. Trokielewicz et al. showed that high quality images acquired with an iPhone 5s can be successfully used with existing commercial and academic iris matchers, with FTE of 0%, and close-to-perfect correct recognition rates, and introduced a dataset of high quality iris images [Tr16]. Crossspectral iris recognition between mobile phone samples and typical NIR images is studied in [TB16], with EER 2%. Handwritten signatures. With advantages such as ease of use, familiarity, and social acceptability, signature biometrics has the potential of mobile use. On a smartphone, one can acquire on-line handwritten signatures (X,Y) coordinates in time, as well as pressure (selected phones). On-line signature recognition methods can be divided into two groups: nonparametric and parametric. The most popular method from the first group is Dynamic Time Warping (DTW) [Bl14,PL16], which normalizes signatures and matches the two samples. EERs achieved in experiments with drawing tablets ranged from 0.6 % to 3.4% for random forgeries and from 5.4% to 17.18% for skilled forgeries. The most common parametric method are Hidden Markov Models [VGSD07,To15]. These methods determine the probability of sample belonging to the learned model of transitions between observation states. EERs range from 4.6% to 7.3% for skilled forgeries, whereas the average for random forgeries EER 3%. Face. In 2012, the bi-modal MOBIO database was published [Mc12], including experiments for face and speaker biometrics running on a Nokia N93i smartphone. Fusing both characteristics resulted in EER 10%. Half error rates were achieved using the same database and a range of different methods in the ICB 2013 Face Recognition Evaluation [Gu13]. Mobile face recognition with locally executed implementations has recently gained interest due to the iPhone X, which employs 3D face recognition. Voice. Due to its convenience in some applications and increasing accuracy, text-independent speaker recognition can be a natural choice for mobile authentication systems. The bimodal MOBIO database [Mc12], contains a collection of voice samples captured with a Nokia N93i. This database was also used in the ICB 2013 Speaker Recognition Evaluation [Kh13], resulting in encouraging error rates of 5% EER. Hand. Hand images can supply information on both hand geometry and texture. The authors of [Sa09] developed a biometric recognition method based on features extracted from hand silhouette and its contour, obtaining EER=3.7%. According to the works of Zhang et al. and Sun et al. [Zh03,Su05], large areas of the palmar side of the hand provide enough personal information for the identity authentication, even with low-resolution images of less than 100 dpi. Hand could be a useful for biometric authentication thanks to simple acquisition process using the built-in camera on a mobile phone without an additional sensor.

Database characteristics
The MobiBits multimodal mobile biometric database includes data collected from 53 volunteers (20 female and 33 male), using three different smartphones. The age of subjects ranged from 14 to 71 years. The data were collected in 3 sessions organized during 3 following months. Data collection took place twice in each acquisition session, separated by approximately 15 minutes. All samples were acquired in typical office conditions with air conditioning set to 24 o C. Other factors that might have influenced the results, such as gender, age, health condition, environmental conditions before the measurement, time from last meal, were recorded in the metadata. For further information on how to get access to the data please contact the authors at: mobibits@nask.pl.
Three smartphones equipped with custom data acquisition software and different technical specifications and capabilities were used for data collection: Huawei Mate S, Huawei P9 Lite, and CAT s60, with the following accessories: an Adonit Dash 2 stylus for signatures to get a more precise contact with the display, hot pillows, and cooling gel compresses were used to simulate different environmental conditions. IRIS AND PERIOCULAR. Rear camera (13 Mpx) of the Huawei Mate S device has been used to capture images of irises (Session 1) and face focused on the eyes region (Session 2 and 3). Data collection was carried out in a typical office setting with artificial light sources. Flash was enabled during the first and the third session, and disabled during the second session to simulate different, real-world conditions of authentication.
HAND IMAGES. For hand data, images of palmar and dorsal side of both left and right hand were collected. CAT s60 phone with built-in thermal camera was used for the image acquisition. According to the current state of the art, thermal images have never been used in mobile biometrics. Due to this fact, hand images were collected according to scenario which allows the temperature dynamics analyses to stimulate rich research ideas regarding this fascinating, yet largely unstudied biometric characteristic. We collected hand images: supported by glass stand and without, after cooling, after warning and with no temperature influence. One session -Extra Session, used 13 Mpx camera, for more accurate images of the hand texture.

Database evaluation: tools, protocol, and performance metrics
For the evaluation of the MobiBits database we performed benchmark calculations of the recognition accuracy for all characteristics, using a proprietary PLDA/i-vector solution for voice recordings, DTW method for signatures, VeriLook for face images, IriCore for the iris images and VGG-based CNN approach for hand images. Dynamic Time Warping (DTW) algorithm calculates an optimal warping path between two given time series is calculated, and then used for comparison [PL16]. VeriLook is a face recognition technology offered by Neurotechnology [Ne17], which uses a set of robust image processing algorithms, including deep neural networks. It allows face recognition with non-frontal images, however, the default allowed rotation angles are modest. IriCore is an iris recognition SDK developed by IriTech [Ir13], with the exact algorithm not disclosed. As this method is fine-tuned to work with iris images compliant with the ISO/IEC 19794:2011 standard [IS11], pre-processing of photos was performed with 640 × 480 cropping and grayscale conversion of the red channel, which is said improve the visibility of the iris texture in heavily pigmented irises [Tr16,TB16]. PLDA/i-vector voice recognition is our proprietary solution based on state-of-the-art PLDA method [PE07,GREW11] with 40 mean and variance normalized MFCC features extracted from every signal frame, a 256-component, gender-independent background model, 64-dimensional total variability subspace, and 64-dimensional PLDA model. The results were normalized using cohort symmetric score normalization [KCD11]. Prior to feature extraction, additional spectral subtraction voice activity detection was performed to remove non-speech periods [Ma10].
The system was trained exclusively on the MOBIO dataset. VGG-16-Hand is a method based on the well-known convolutional network VGG-16 model [SZ14], with weights finetuned on a dataset of hand images from the Extra Session with flash enabled (cf. Section 3), and its bottleneck layers modified to reflect the number of output classes.
IriCore and VeriLook engines performed quality checks prior to feature extraction and biometric template creation. Since some samples were not accepted, failure-to-enroll rates (FTEs) were calculated to express the ratio of rejected samples to all samples. Each biometric characteristic and each fusion model was tested in a verification scenario. Averaged Receiver Operating Characteristics (ROCs) and mean Equal Error Rates (EERs) were used to present the verification accuracy. These performance metrics were calculated for comparison scores obtained by performing all possible comparisons within the data, excluding the within-session ones, as these would involve comparing highly correlated data.
Please note that thermal images of hands and those of faces are not used for comparisons here, as we are not aware of any recognition methods that would be capable of efficient processing of such samples. Instead, our intent was to provide them as datasets that are complementary to these of visible light images, so that they can be utilized for other research, such as designing robust methods for Presentation Attack Detection (PAD).

Results and discussion
FTEs calculated for all tested characteristics are presented in Table 2. High FTE values for the face engine may be explained with difficulties in detecting and processing profile images by the VeriLook software, as the profile and semi-profile images constituted around 60% of the total image count. Iris enrollment errors can be a result of using mixed quality grayscale images converted from RGB images, instead of high quality NIR images recommended for the IriCore software. Other problems may be related to: iris image noise, reflections on the eye, lashes, squinted eyes. The voice recognition engine, the CNN for hand images and the DTW method did not perform any quality checks, hence zero FTEs.
The ROC curves together with respective EER values are shown in Figure 2. The best single characteristic results can be achieved with face recognition, which allows verification accuracy with EER=2.32%. The worst result was obtained for signatures -skilled forgeries, EER=18.44%. High equal error rate for the irises is related to differences in the quality of the pictures, as high quality pictures obtained with flash enabled are compared against low-quality, no-flash pictures.

Conclusions
The most important deliverable of this paper is the multimodal mobile biometric database -MobiBits, comprising samples of five different biometric characteristics, collected using three different mobile devices, including a novel type of samples: thermal images of hands and faces, acquired using a thermal sensor equipped mobile phone, which we think can be a valuable contribution for studies involving presentation attack detection methods. A comprehensive set of experiments conducted on the data is reported to show the example benchmark accuracy that can be achieved on the dataset using selected commercial and academic recognition solutions. The MobiBits database is offered publicly at no cost for non-commercial research purposes. We hope that this will constitute a worthy addition to the mobile biometric datasets available for the biometrics community, and will provide an incentive for further research in the field of mobile biometrics.