{'fraction_non_alphanumeric': 0.0352363960749331, 'fraction_numerical': 0.009455842997323818, 'mean_word_length': 5.13293216630197, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
State of the Art: Face Recognition

Working with Child Sexual Exploitation Material (CSEM) in forensic applications might be benefited from the progress in automatic face recognition. However, discriminative parts of a face in CSEM, i.e., mostly the eyes, could be often occluded to difficult the victim's identification. Most of the face recognition approaches cannot deal with such kind of occlusions, resulting in inaccurate face recognition results. This document presents a short review face recognition methods for images with natural and eye occlude faces. The purpose is to select the best baseline approach for solving automatic face recognition of occluded faces.


Introduction
Face recognition is one of the most broadly researched topics in academic and industrial fields, due to its extensive applications in law enforcement and surveillance, information security, access control, smart cards, and others. In recent years, Deep Convolutional Neural Networks (CNNs) based methods have improved performance significantly [1,2,3,4].
Fast detection of Child Sexual Exploitation Material (CSEM) could prevent its distribution as soon as possible and would allow building a legal case against presumed offenders.
However, manual detection of such material is time-consuming and disturbing for LEA operators.
In the context of forensic tools, the automatic detection of CSEM [5] represents a substantial assistance to Law Enforcement Agencies (LEA). In these tools, face recognition can support the task of victim identification in CSEM, apart from establishing links between different CSEM cases [6].
However, in CSEM, it is frequent that the offenders manually occlude the victim faces to difficult their identification, e.g. Figure 1, which will pose a challenge to face recognition algorithms [7]. Under these occluded conditions, the performance of face recognition algorithms drops, mainly when a mask covers the eyes, an object, or by an adversarial attack [7]., which is an image modification whose intention is to alter or perturb an image to fool a classifier [8].
Some approaches to handle the occlusion challenge in the face recognition task have been presented in the last few years. There are frameworks based on auto-encoders [9,10], that remove the occluded parts of the face prior to the recognition. Other approaches have proposed local feature learning-based methods, such as constraints-based dictionary learning [11]. Lastly, some approaches have focused on the unbalance between negative and positive samples, such as sparse coding with manifold learning [12], or kernel prototype similarities [13].

Face Recognition
Nowadays, significant development has been achieved in face recognition research [1,3,14], mainly due to the availability of massive data and Graphical Processing Unit (GPU)s to train Deep Learning models. Some existing approaches [15,16] have also focused on face recognition with expression, pose, aging, disguise, or illumination changes.

Traditional Face Recognition Methods
Several traditional face recognition methods have been found in the literature, and one of the first approaches has been proposed by Ojala et al. [17] using Extended LBP (ELBP) to improve the discriminative capability of faces. The ELBP generates a binary comparison between the central pixel and its neighbors as well as encodes their exact grey-value differences using some additional binary units.
Wang et al. introduced stationary wavelet entropy to extract features and employed a single hidden layer feedforward neural network as a face classifier [18]. Moreover, they also introduced the Jaya algorithm to prevent the training of the classifier fall into local optimum points.
Model and geometry-based methods are other strategies to recognize a face in uncon- Multi-Dimensional Scaling (MDS) method to learn a mapping matrix, which projects the high-resolution and low-resolution face images to a common subspace [20]. They add an inter-class constraint to enlarge the distances of different subjects in the subspace to ensure discriminability.

Face Recognition using Deep Learning Methods
The previous methods are all based on hand-crafted features, and those features may fail to capture important feature information to discriminate against the faces. Nowadays, deep learning-based approaches have become more popular because those methods can deal with large training datasets by learning the rich and compact representation of the face, such as FaceNet [14], VGGFace [21], and DeepFace [22]. Georgescu et al. proposed a hybrid approach that combines the features learned by CNN and hand-crafted features calculated using bag-of-visual-words (BOVW) [23]. Lastly, Lu et al. [24] introduced a deep coupled ResNet (DCR) model for low-resolution face recognition. This model consists of two small branch networks and a big trunk network. The trunk and branch networks have been trained to learn discriminant features shared by face images of different resolutions, and to learn resolution-specific coupled-mappings (CMs), respectfully. They projected the high-resolution gallery images and low-resolution images to space, where their distances were minimized, to recognize the face.

Face recognition against Occlusion
As previously mentioned, occlusion is considered one of the most challenging problems in face recognition. Both deep CNN-based or traditional face recognition methods cannot function well against occlusion due to large intra-class variation and higher inter-class similarity.
Apart from Deep learning-based methods, face recognition against occlusion has been handled using different approaches. For instance, Morelli Andrés et al. [25] employed compressed sensing to detect the occluded part from the face image and then remove it.
They used local features to generate a new, non-occluded image that is similar to the one they attempted to recognize. A query image was then subtracted from this new image to detect the occlusion area through a threshold. In the end, only the non-occluded pixels were used to recognize the identity.
Dagnes et al. [26] proposed a method for 3D face recognition, robust to the eye and mouth occlusions. These obstructions were detected and removed by exploiting the 3D geometry, i.e., by considering their effects on the 3D points. Lastly, the non-occluded symmetrical regions were used to restore the missing facial information prior to recognizing the face.
Domingo et al. [27] represented the query and the gallery images by means of random patches, described by their location and intensity information. These patches were used later to build a dictionary. Wu et al. [28] proposed a method, called Occlusion Pattern-based Sparse Representation Classification (OPSRC), to learn the occlusion pattern from the query data. Mustafa et al. presented an occluded face recognition framework [29] based on the two-dimensional Multi-Color Fusion (2D-MCF) representation and the Partitioned-sparse sensing recognition (P-SRC) classifier.
Local feature learning is another approach to deal with occlusion. In this procedure, features are extracted from local areas of the face image, and they are used for the recognition through a locally matching strategy. Based on this concept, Liao et al. [30] presented the Multi-Keypoint Descriptors (MKD) to represent the alignment-free face where the actual content of the image determines the size of the descriptor.
To recognize the partially occluded face, Duan et al. [31] proposed a scheme based on topology-preserving graph matching to estimate more accurate and robust topological information. It has estimated a non-rigid transformation encoding the second-order geometric structure of the graph.
Non-negative matrix factorization (NMF)-based learning provides an effective way for face recognition robust against occlusions. An example is the dictionary learning method proposed by Ou et al. [32]. They created low-dimensional representations of samples from the same class to be as close as possible to enhance the discriminant ability of the dictionary.
On the contrary, an LSTM-autoencoders model was introduced by Zhao et al. [33] which consists of a multi-scale spatial LSTM encoder to generate an occlusion-robust representation of the face, and a dual-channel LSTM decoder to recurrently remove the occlusion in the image space.
Additionally, several recent approaches have addressed the occlusion problem by employing low-rank representations [34], hierarchical sparse and low-rank representations, [35], a discriminative multi-scale sparse coding (DMSC) [36] and fuzzy max-pooling to solve the double-occlusion problem [37]. Table 1 shows the face classification experiment with the datasets, Labeled Faces in the Wild (LFW) [38], which has 13, 233 images belonging to 5, 749 identities, and Celebrities in Frontal-Profile in the Wild (CFPW) [39], which has 7, 000 images of 500 identities. Note that we created eye occluded versions of both datasets by occluding the eye region of their faces. In face classification against occlusion, embeddings of natural faces using CNNs models, i.e., Dlib [40], VGG16 [21], OpenFace [3], FaceNet [14], and ArcFace [1] were extracted as well as the face features were computed using the OSF-DNS hashing method

Discussion and Conclusions
In this document, we have presented a study of the state-of-the-art traditional and deep learning-based methods for recognizing a face from natural images. Then, we have pre- It is observed in Table 1 that the classification accuracies of all deep feature-based methods are generally poorer than the perceptual hashing method, OSF-DNS, for both datasets. The reason may be that occlusion leads to a distortion of the face embeddings obtained by the convolutional base of the networks, which makes discrimination more difficult. Mainly, ArcFace attains much worse performance than the rest of the CNN-based approaches. In ArcFace, a more reliable method to increase the feature distances is applied [1]: an arc-cosine function is applied in the angular domain so that the decision bound-aries between features corresponding to different classes are more distant from each other.
During the experiment, we mainly extracted the embedding features of a face and its eye occluded version through the pre-trained ArcFace model. Though both faces are the same except the occluded region (i.e. they belong to the same class), the method provides very different embeddings for them. The reason may be that in ArcFace, the embeddings of the faces are distributed around each feature center toward the hyper-sphere and it uses an additive angular margin penalty between feature and ground truth weight to concurrently enhance the intra-class compactness and inter-class discrepancy.
In this work, however, the pre-trained model is used, which is trained with non-occluded faces, to extract the embeddings of an occluded face. Therefore, occluded face features for the same identity are distorted, thus affecting the accuracy.
Lastly, the revision of this document concludes that traditional deep learning-based approaches may not perform well in the task of face classification or recognition against oc-