{'fraction_non_alphanumeric': 0.027239367707941286, 'fraction_numerical': 0.008844561535566429, 'mean_word_length': 5.222775175644028, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Deep Learning Modalities for Biometric Alteration Detection in 5G Networks-Based Secure Smart Cities

Smart cities and their applications have become attractive research fields birthing numerous technologies. Fifth generation (5G) networks are important components of smart cities, where intelligent access control is deployed for identity authentication, online banking, and cyber security. To assure secure transactions and to protect user’s identities against cybersecurity threats, strong authentication techniques should be used. The prevalence of biometrics, such as fingerprints, in authentication and identification makes the need to safeguard them important across different areas of smart applications. Our study presents a system to detect alterations to biometric modalities to discriminate pristine, adulterated, and fake biometrics in 5G-based smart cities. Specifically, we use deep learning models based on convolutional neural networks (CNN) and a hybrid model that combines CNN with convolutional long-short term memory (ConvLSTM) to compute a three-tier probability that a biometric has been tempered. Simulation-based experiments indicate that the alteration detection accuracy matches those recorded in advanced methods with superior performance in terms of detecting central rotation alteration to fingerprints. This makes the proposed system a veritable solution for different biometric authentication applications in secure smart cities.


I. INTRODUCTION
Use of human biometrics in identity authentication is an area of interest for researchers from different backgrounds. Moreover, the identity of a person can be authenticated via different biometrics such as finger vein, face, fingerprint, and The associate editor coordinating the review of this manuscript and approving it for publication was Zahid Akhtar .
iris. Human identification is deployed in several applications such as access control, cyber security, and blockchains. Furthermore, owing to their uniqueness and animateness, more recently, the human biometrics are replacing passwords in several applications. For example, smart phones use the iris and fingerprint to be the password to access the device. Additionally, some financial organisations like Paneer have provided mobile applications for money transfer via the use of fingerprints for the client authentication. Furthermore, the advent of online banking and its pervasiveness have led to increases in attempts to hack bank accounts via password spoofing as well as efforts to alter the biometric of the person. As the most common biometric used in person identification, verification and other cyber security applications, fingerprints are frequently targeted by criminals. Consequently, it is important to determine whether the integrity of such biometric data has been impugned.
In this regard, various studies for authentication detection prior to recognition of biometrics have been undertaken. Among them, Gad et al. [1] proposed a unimodal recognition methodology based on iris as well as multi-biometric scenarios. They proposed new modalities in both segmentation and feature extraction phases. In the segmentation phase, they proposed a masking technique for localisation and two modalities, (delta mean, and multi-algorithm mean) which they validated using CASIA v.1, CASIA v.4 and UBIRIS v.1. In [2], Wang et al. proposed an iris recognition technique based on Navier-Stokes (NS) equation with 1D Gabor filter used to construct the feature vector. They reported implementing their proposed technique using iris images exposed to varying levels of noise. Their experiments validated the techniques capability to accomplish person recognition despite the iris biometric being exposed to different noise conditions.
Other areas of use of iris recognition are in person authentication and identity verification. For example, in [3], Gad et al. replaced the traditional use of username and password with the iris as a unimodal biometric trait. The feature extraction stage of their proposed scheme utilises the Delta Mean (DM) and Multi-Algorithm Mean (MAM) that is followed by a feature reduction process to guarantee good performance. The scheme also has a classification stage that is undertaken using a Euclidian Distance (ED) classifier.
Similarly, authentication and identification suing finger vein have also been proposed. In [4], Peng et al. proposed a finger vein recognition method using a combination of Gabor wavelet and Local Binary Patterns (LBPs) with a Block-based Linear Discriminant Analysis (BLDA) for the task of feature reduction. Another finger vein recognition approach based on image quality enhancement was proposed by Peng et al. [5]. They used a Region of Interest (ROI) methodology for finger vein segmentation. Furthermore, the finger vein method was deployed for authentication in [6]. Specifically, Gabor filter was used to select parameters in eight orientations of the finger vein network, whose patterns are extracted using two distinct orientations. For their contribution in [7], Peng et al proposed a multi-biometric authentication system based on variations of fingerprints including finger knuckle, shape, vein as well as the original fingerprint of a certain person. A score-level fusion modality based on triangular norm on these fingerprints is then used to generate a single image representing the person's identity. Finally, they validated their system on a merged dataset.
Face biometric is involved in the face recognition process in several applications. In one such application in [8], a fusion process is performed using Multiscale Complex Fusion for Gabor Jet Descriptor (MCFGJD). The main idea of this work is to extract the features from the images using the Gabor descriptors generated from some block-wise statistics of the image including mean and standard deviation, which are generated from the magnitude, phase, real and imaginary parts of Gabor coefficients. After that, a feature reduction process is carried out on the extracted features using Linear Discriminant Analysis (LDA) algorithm. Finally, both thermal and visual images are fused.
Another research trend in biometric-based human identification involves a combination of two or more biometrics to improve security and authorisation. For example, the work in [9] presented a human identification system based on both thermal and visible face images as well as iris. This work depends on 1D Log-Gabor filter to generate a single feature vector from the dual iris codes. In addition, complex Gabor Jet descriptors are extracted from the thermal and visible face images to enhance the representation of their features. Furthermore, an image fusion technique is used to combine all biometrics. In [9], an Aczel-Alsina triangular norm (AA t-norm) technique was used for image fusion. In this way, the system can be used in both identification and authentication applications.
Meanwhile, the potency of Deep Learning Models (DLMs) has led to its deployment in a wide range of applications comprising medical image diagnosis [10]- [12], forgery detection [13], etc. Our proposed system uses a convolutional neural network (CNN) and a hybrid structure that combines CNN with convolutional long-short term memory (ConvLSTM) as DLMs for the detection of different levels of alteration in biometrics that are employed for person identification.
The remainder of this study is organised as follows. The outlines of our proposed system are presented in the next section (i.e., Section II), and the foundations of the proposed DLMs are discussed in latter parts of the same section. Experimental validation of our system as an effective platform for detection of alterations in biometric data is reported with discussions in Section III and IV, respectively.

II. PROPOSED FRAMEWORK AND DEEP LEARNING MODELS (DLMS)
We envision the deployment of our system as part of applications in a typical smart city, such as online banking platforms that are important for financial inclusion like electronic wallets. Like many of the emerging technologies, our system seeks to replace traditional username and password credentials with unique biometric traits of individuals. Despite enhancing secure access and authentication of users, biometrics are susceptible to the unrelenting efforts of criminals, such as spoofing to alter the biometrics. Therefore, it is necessary to discriminate between pristine (i.e., original) and altered biometrics such as fingerprints.  Figure 1 presents an overview of the conceptual platform we envisage for our proposed framework. Its implementation presupposes a client using his mobile device for online banking applications, where the client's fingerprint biometric is used for authentication and verification in financial transactions. In their illicit effort to violate such systems, criminals alter the biometric data which could be deemed at one of three levels (easy, medium, and hard).
This makes efforts to discriminate pristine and altered biometric modalities crucial for financial transactions in smart cities and Internet-of-Things (IoT) platforms. Developing a deep learning framework to accomplish this task is the main objective of our study. Details of our proposed system are presented in the remainder of this section.
Deep Learning Models (DLMs) depend on multiple layers to progressively extract higher-level features from unstructured data. Its use in our proposed biometric alteration detecting system is accomplished via three stages as illustrated in Figure 2. In the first stage, a pre-processing procedure is performed on the input images to unify their sizes and convert them into tensors suitable for DLM processing. In the second (i.e., extraction) stage, the features from the images are transformed using a hierarchy of convolutional (CNV), Convolutional Long Short-Term Memory (ConvLSTM), pooling, Global Average Pooling (GAP) and decision making (dense) layers. Additionally, activation functions, such as the Rectifier Linear Unit (ReLU) and SoftMax functions are used to evaluate the output of each layer. This study presents two DLMs. The first model is based a CNN consisting of three CNV layers followed by three pooling layers. This sequence of convolutional and pooling layers is required to extract the features from the input image tensors. Furthermore, the extracted feature map of each layer serves as input to an activation function to obtain the output of this layer. The activation function of each convolutional layer is the ReLU activation function, whose positive values are obtained with the same values, while the negative values are set to be zero at the output of the activation function as defined in Equation (1).
Another feature extraction process tailored towards feature reduction is carried out by pooling layers. In this regard, there are two main types of pooling process. The first type, the Max pooling, is required to extract the maximum value of each block, whereas the second type, i.e., the average pooling, extracts the average value of each block in the feature map as depicted in Figure 3. The architecture of the CNN DLM in Figure 4 outlines a data flow, where input layers are unified into a 224×224×3dimensional input. The number and type of filters used determine the dimensions of the output feature map. For example, a model whose first convolutional layer has 16 filters and a stride of 2 would produce a 222 × 222 × 16 output feature map. The second model utilised in our study consists of a ConvLSTM layer followed by two convolutional layers each of which is followed by a pooling layer for feature reduction. The architecture and data flow of this model is presented in Figure 5. Additionally, as depicted in Figure 4, the shape of the output feature map of the pooling layer is reduced in two dimensions only.
The Long Short-Term Memory (LSTM) is a special case of the artificial Recurrent Neural Networks (RNNs), which like the standard RNN, is stable and reliable for long-range dependencies [12]. Despite this, the LSTM is impeded by redundancy of spatial data. The ConvLSTM augment this shortcoming by replacing fully-connected layers in the LSTM with convolutional layers [12]. In this structure, ConvLSTM evaluates the same functionality of LSTM in 2D images [13]. Like LSTM, in ConvLSTM, the current state depends on all previous states as depicted in Figure 5 [12] and formulated in Equations (2) to (6).
Consequently, it can be inferred that if states are viewed as they would in the hidden representations of moving objects, then the larger transitional kernel of a ConvLSTM should be capable of capturing faster motions [12]- [18]. Furthermore, ConvLSTM has the ability to encode the spatio-temporal information in its memory cells. To ensure that the states have the same dimensions as the inputs, padding is needed before applying the convolution operation. Furthermore, all states are initialised to zero before the first input, which corresponds to the so-called total ignorance of the future [12].

III. SIMULATION EXPERIMENTS AND RESULTS
This section presents outcomes of extensive simulation-based experiments undertaken to prove the efficiency of our proposed system in detecting alterations to fingerprint biometrics employed across different applications for person identification and verification.
We begin with an overview of the dataset used in the experiments; definitions of metrics employed for our quantitative validation of the proposed system as basis for comparisons with state-of-the-art techniques obtained from similar studies.
where: i t : The input gate. The simulation experiments are carried using the Sokoto Coventry Fingerprint (SOCOFing) dataset [18]. SOCOFing dataset includes 6,000 fingerprint images from 600 persons, i.e., 10 fingerprints for each person. In addition, SOCOFing dataset includes three different alteration methodologies: obliteration, central rotation, and z-cut. Each alteration method consists of 15,000 altered images. Therefore, SOCOFing dataset consists of 50,000 images for both pristine and altered fingerprint images. Figure 6 shows samples of the fingerprints for a certain person.

B. EVALUATION METRICS
Our proposed system is evaluated using standard metrics as employed in recent and established studies. The equation matrix [12] in Figure 7 summarises definitions for specificity, accuracy, Positive Predictive Value (PPV), Negative Predictive Value (NPV), F1 score and Mathew's Correlation Coefficient (MCC). Based on standard definitions, True Positive (TP) is the number of tampered images that were truly detected as tampered images, True Negative (TN) is the number of original images that were truly detected as original images, False

C. SIMULATION RESULTS
The experiments reported in this section were simulated on an Intel core i7 8th edition CPU workstation with NVIDIA 4GB DDR5 GPU and 16GB RAM DDR5.
The objective of experiments is to ensure the effectiveness of the proposed system in detecting the alterations that may occur in the fingerprints during authentication processes encountered in typical online banking transactions. As presented earlier in Section III, our study depends on the use of two deep learning frameworks (i.e., CNN and a hybrid one that combines ConvLSTM and CNN) for efficient detection of alterations to fingerprint of authorised users. Furthermore, the validation reported in this subsection employs fingerprint images obtained for three levels of alteration (i.e., obliteration, central rotation, and z-cut) as available in the SOCOFing dataset [19], [20].

1) RESULTS FOR OBLITERATION ALTERATION DETECTION
The obliteration alteration occurs, when the attacker destroys traces of some areas of the fingerprint to spoof the original fingerprint of a certain person. This type of alteration is considered as a hard alteration, because the outcome imbues characteristics far from those of the original fingerprint. Therefore, this type of alteration is easy to be detected. Figures 8 and 9 present accuracy and loss curves of obliteration alteration detection using the CNN DLM, while similar results for the hybrid DLM are presented in Figures 10 and 11. Furthermore, Table 1 presents evaluation outcomes for the obliteration alteration detection with both models. From the table, we notice that an average detection accuracy of 92.97% is recorded for the CNN DLM and 93.88% is recorded for the hybrid ConvLSTM model.

2) RESULTS FOR CENTRAL ROTATION ALTERATION DETECTION
The central rotation alteration is considered as a medium alteration that is based on rotation operations on the   Table 2 presents the outcomes of evaluation for central rotation alteration detection using both models. As reported in this table, the CNN DLM records an accuracy of 98%, whilst the hybrid DLM attains an accuracy of 99%.

3) RESULTS FOR z-CUT ALTERATION
The proposed DLMs have been implemented on the z-cut altered fingerprint dataset [19]. Figures 18 and 19 present plots of accuracy and loss curves for z-cut alteration detection using the CNN DLM. Similarly, Figures 20 and 21 present plots of accuracy and loss curves for z-cut alteration detection using the proposed hybrid model. Furthermore, Table 3 highlights the performance of both DLMs in z-cut alteration detection. As reported in this table, the proposed CNN DLM   records a detection accuracy of 98.76%, while the hybrid DLM attains an accuracy of 99.05%, which confirms the effectiveness of both models.

IV. DISCUSSION OF RESULTS
With an average detection accuracy of 97%, the results recorded in the previous section illustrate the effectiveness of both DLMs in our proposed system as tools to detect alterations to fingerprint biometrics. To further establish this performance, in this section we discuss the recorded results relative to results of a state-of-the-art technique reported in the literature.   In [20], CNN and ResNet18 DLMs were utilised for detection of fingerprint alteration, the CNN structure was made up of convolutional layers with 20 3 3, 40 3 3, 60 3 3, and 80 3 3 kernels. Additionally, the convolutional layers were followed by two fully-connected layers with 1000 and 100 hidden units, respectively. In another model, ResNet18 pretrained model was used. Table 4 highlights the performance of our proposed DLMs alongside those reported in [20] in terms of alteration detection accuracy and Predictive Positive Values (PPV).   As seen from the table, both systems report acceptable levels with our system beaten to second place in terms of PPV for obliteration alteration detection, while it tops in terms of PPV for the central rotation and z-cut alterations. Furthermore, our proposed DLMs perform better than [20] in terms of detection accuracy for the central rotation alteration. These results indicate the ability of our proposed models to match established ones with superior performance for central rotation alteration detection.

V. CONCLUDING REMARKS
With the advances in intelligent systems engineering, smart technologies will become ubiquitous in emerging smart cities. At such stages, fifth generation (5G) networks will require technologies to mitigate identity theft, enhance person identification and other Intelligent Access Control (IAC) mechanisms.
In tandem with these advances, biometric identification will become pervasive. Our study is primarily aimed at providing additional security to these platforms.
The prime challenge in such IAC systems is the ability to efficiently discriminate between the original and fake biometrics, where depending on sophistication, efforts to impugn the integrity of biometric markers could vary from easy to medium to high. Our study presented an alteration detection system using deep learning models based on CNNs and Con-vLSTM. The proposed models are implemented on a dataset of fingerprint images comprising three types of alterations. Outcomes of reported simulation-based experiments ensure the effectiveness of our proposed system in the detection of fingerprint alterations with superior performance, which makes it viable for applications in 5G network-based secure smart cities.  He is also a Professor with the School of Computer Science and Technology, Changchun University of Science and Technology, China. In addition to being among the pioneers of research in the emerging quantum image processing (QIP) subdiscipline, he has to his credit more than 150 publications traversing the areas of computational intelligence, quantum cybernetics, quantum image processing, quantum machine learning, cyber and information security, intelligent systems engineering, the Internet of Things, 4IR, health informatics, and electronic systems reliability. As well as being the Managing Editor of Fuji Technology Press, Japan, he is a member of Editorial Board of the Journal of Advanced Computational Intelligence and Intelligent Informatics (JACIII), Quantum Reports journal, International Journal of Electrical and Computer Engineering (IJECE), and the Journal of Medical Imaging and Health Informatics (JMIHI). He is also an Associate Editor of many other journals, including IEEE ACCESS, Telkomnika, and Information Sciences.