{'fraction_non_alphanumeric': 0.03190953623586504, 'fraction_numerical': 0.014695783546216179, 'mean_word_length': 4.808029779314012, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Comparing Performance of Supervised Learning Classifiers by Tuning the Hyperparameter on Face Recognition

In this era, face recognition technology is an important component that is widely used in various aspects of life, mostly for biometrics issues for personal identification. There are three main steps of a face recognition system: face detection, face embedding, and classification. Classification plays a vital role in making the system recognizes a face accurately. With the growing need for face recognition applications, the need for machine learning methods are required for accurate image classification is also increasing. One thing that can be done to increase the performance of the classifier is by tuning the hyperparameter. For this study, the evaluation performance of classification is conducted to obtain the best classifier among four different classifier algorithms (decision tree, SVM, random forest, and AdaBoost) for a specific dataset by tuning the hyperparameter. The best classifier is obtained by evaluating the performance of each classifier in terms of training time, accuracy, precision, recall, and F1-score. This study was using a dataset of 2267 facial data (128D vector space) derived from the face embedding process. The result showed that SVM is the best classifier with a training time of 0.5 s and the score for accuracy, precision, recall, and F1-score are about 98%.


Introduction
A face is a key to the human perception system [1]. Different facial cues can identify different people. The most important facial cues include nose, eyes, jaw, hair, and mouth [2]. The advances in technology, the development of existing hardware and software, allows faces to be processed into objects or often known as face recognition. Face recognition has been used for many applications, which are for security, image database investigation, face identification, access control, video surveillance, generally identify verification, smart card application, the criminal justice system, shopping sense like in Amazon Go, etc [3,4,5,6,7].
The general process of face recognition consists of face detection, face embedding, and classification. The first process of face recognition is started by detecting whether there is a face or not in a picture/frame. This is done by identifying the sizes of a known number of faces and their locations [8]. If the face has been found in a frame, then the face embedding will be performed. Face embedding acts as a feature extractor from the detected faces. The goals of this process are to reduce the machine training time and the space complexity [9]. Feature extractor transforms an input into a set of features and selects a feature containing the most relevant information [10]. The selected features are then classified by the classifier. Generally, this process consists of a machine learning algorithm that has been trained with a number of certain datasets so it can obtain the desired criteria.
The very important part of the face recognition process is classification because it is a learning process that allows the system to differentiate the classes by reaching the minimum possible error based on the training dataset to obtain accurate facial recognition. Face classifications are frequently performed using a supervised learning algorithm due to Copyright © 2021 MECS I.J. Intelligent Systems and Applications, 2021, 5, 1-13 high-dimensional data. Image classification can be done by using some kind of supervised learning algorithms such as support vector machine (SVM), random forest, AdaBoost, decision tree, etc. Each algorithm is modeled in such a way so it has certain different hyperparameters. This hyperparameter affects the performance of the classifier in building a robust and accurate model [11,12], hence, the optimal hyperparameters are needed to obtain maximum performance. Previous studies have been comparing several classifiers on face recognition to find the best classifier. Emir et al., Abdel et al., and Huda et al. [13,14,15] have been comparing SVM classifier and random forest classifier on face recognition and using Histogram of Oriented Gradients (HOG) as the feature extractor. They got a result that random forest reached higher accuracy than SVM with the accuracy value is 97.2% [13], 95.1% [14], and 95.9% [15]. Nurbaity et al. [16] are also comparing naï ve Bayes, SVM, and multi-linear perceptron (MLP) classification. The result showed that naï ve Bayes has higher accuracy than the others with a value of 93.2%. Wang et al. [17] compared Adaboost, SVM, linear logic classification, nearest neighbor algorithm (KNN), and linear discrimination (LDA) and the result is linear logic got the highest accuracy among the others with the value is 99%. Last, Nabatchian et al. [18] tested the performance of KNN, SVM, and hidden Markov model (HMM) classifiers, and the best accuracy was reached by HMM with the value of 91%.
This study also evaluates several different classifier algorithms for a specific dataset by evaluating the performance of each classifier to obtain the best classifier by tuning its hyperparameters which most studies have not done before because tuning the hyperparameter would be a tedious and continuous task to randomly trying the hyperparameter values, whereas hyperparameter can improve the performance of the classifier. Hyperparameter tuning is also tricky in the sense that there is no direct way to calculate how a change in the hyperparameter value will reduce the loss of the model, so tuning a model is trial-and-error based engineering.
The used dataset contains 2267 facial data (128D vector space) derived from the face embedding process. In this paper, we conducted data pre-processing, model validation, hyperparameter tuning, and model evaluation on four classifiers: decision tree, SVM, random forest, and AdaBoost. This study also enriches a reference to the comparison of the classifier algorithm that developers can use in choosing the best classifier algorithm according to their needs to obtain an optimum face recognition system. Besides the accuracy level, this research also evaluates the performances in terms of training time, precision, recall, and F1-score. And the result showed that the best classifier is SVM with a training time is 0.5 s and the score for accuracy, precision, recall, and F1-score are about 98%.
This paper consists of five sections. The first is an introduction that discusses an overview of the face recognition process, particularly for classification parts, and discusses previous studies and the background of this research. The second section is a literature review which tells us about literature related to this research. The third discusses the methodology which consists of a dataset and experimental methods. The fourth discusses the experimental result of the proposed methods. The last part is the conclusion of this research and its future work.

Literature Review
A supervised learning classifier is being applied for many domains, one of them is face recognition. This algorithm predicts the values of a class from labeled data. This algorithm has some hyperparameters that can be tuned manually by configuring them to obtain optimal predictive performance. To be able to adjust the hyperparameters, we need to understand the means of the hyperparameter and the effect on the model, hence, this section discusses a brief explanation about the classifier model that we used for this experiment and its hyperparameter and the effect on the classifier model.

Support Vector Machine (SVM)
Support Vector Machine (SVM) is one of the supervised algorithms that is commonly used for image classification due to SVM performs better than another classifier in terms of less structured data [19]. SVM represents a geometrical model that finds the best margin that differentiates data points of different categories using this function (1) [20]. (1) subject to this constraint in (2): where ( ) being the label for data-point , being the matrix, being the weight vector, and 0 being the bias. The decision boundary of SVM is given in (3).
where is the margin in the decision boundary as in (4).
Copyright © 2021 MECS I.J. Intelligent Systems and Applications, 2021, 5, 1-13 In this experiment, we optimized the hyperparameter of SVM before we trained the models. In SVM, hyperparameter helps to find the balance between the bias ( 0 ) and variance ( ) that will affect the decision boundary ( , 0 ) and thus prevent the model from underfitting or overfitting. SVM has several parameters [21]. In this experiment, we use four different parameters. The parameters are in Table 1.

Decision Tree
The decision tree model has two components, which are nodes and branches, and has three important steps, splitting, pruning, and tree decision. [22]. A decision tree has an issue of robust to irregulates input data and it is insensitive to outliers, it is easily overfitting and unstable [23]. This overfitting can be counteracted by adjusting the hyperparameters. The hyperparameters will stop the recursive splitting process (splitting is a process of partitioning a node into two or more sub-nodes), so it is setting constraints on tree size. There are several parameters for the decision tree algorithm [24,25,26]. In this paper, the experiment uses four different parameters, which are as in Table 2.

Random Forest
Random forest combines decision trees, feature sub-sampling, and bootstrap aggregation to reduce the trade-off in the decision tree by decreases the model variance, so it can increase the accuracy [27]. Random forest collects M randomized trees as denoted in (5) [28].
the -th tree gives the predicted value at point , ⊖ 1 … . ⊖ being independent random variables, and being independent of sample. Random forest prediction is given by the average of the prediction of M randomized trees and it is denoted in (6) In the random forest, several hyperparameters that can be tuned during the training process. The hyperparameters include the number of the randomized tree (M) and the number of independent samples ( ). Several parameters that used in this paper are in Table 3 [29].

Adaboost
Adaboost improves different classification performances by combining the weak classifiers to become a strong classifier with a scheme of weighted majority voting [30]. Adaboost is popular for face recognition applications, for example in the Viola-Jones framework [31]. It utilizes the exponential loss function ( , ( )) = − ( ) , where ( ) being the prediction and being the actual label. The equation for optimizing the cost function is denoted in (7) [30]. Tuning the hyperparameter will affect the accuracy of the models. In the case of Adaboost, tuning the hyperparameters will change the number of weak learners ( ( ( ) , { } )) and control the loss function ( ( ( ) , { −1} ( ( ) )) used for calculating the weight ( { } ). This experiment uses three different parameters as in Table  4 [32] [33]. The weight is applied to each classifier at each boosting iteration. Each classifier contribution can increase by increasing the learning rate value Algorithm 'SAMME', 'SAMME.R' 'SAMME.R' • If 'SAMME.R', use SAMME.R as a real boosting algorithm. SAMME.R has faster performance than SAMME, it has a lower test error with fewer boosting iterations • If 'SAMME', uses SAMME as a discrete boosting algorithm Copyright © 2021 MECS I.J. Intelligent Systems and Applications, 2021, 5, 1-13

Methods
This research aims to find the best performance of a classifier. The general process of the face recognition task can be described by the following list: Face Detection, Face Embedding, and Classification. The experiment was conducted in a workstation with the specification as follows: CPU Intel Core i7-8700 3.2GHz 6 Core 12 Threads, RAM 32GB DDR4 PC-19200, GPU: GeForce RTX 2070.
The first process of face recognition is started by detecting whether there is a face or not in the picture/frame. The algorithm then returned the list of the face(s) and its location in the form of a bounding box(es). Histogram of Oriented Gradients (HOG) is used along with linear Support Vector Machine (SVM) to detect face and its location. HOG is used for this paper because it is a simple and fast algorithm and performs well in detecting face [34] by dividing the image into many cells where a histogram counts the occurrences of pixels' orientations given by their gradients [35]. It summarizes the distribution of measurements within the image regions [36]. In this experiment, we used the HOG library in python.
The next step is face embedding. Face embedding acts as a feature extractor from the detected face(s). Convolutional Neural Network (CNN) with metric loss is used to embed detected face(s) into a 128D vector space. We used CNN because it is a powerful feature extractor that can extract complex data layer by layer and finally form ideal features suitable for classification [37]. This paper also used a CNN library from python.
The vector space from the face embedding process is the final feature used in classification. The details of the classification process will be described in the next subsection. After the classification process, we will evaluate the performance of each classifier to obtain the best classifier. The overview of general methods is shown in Fig.1.

Dataset
The used dataset contains 2267 face data. The face data is a 128D vector derived from the face embedding process with the addition of the label column. There are nine classes of face, consisting of person 1 to 8 plus unknown class. The full dataset is then divided into two separate datasets for training and testing samples by random sampling. The training dataset contains 1813 data while the test dataset contains 454 data. The distribution class of the dataset showed in Fig.2. The training data becomes an input for hyperparameter optimization to obtain the best configuration of hyperparameter that will be used in the final model.

Experimental Methods: Classification Process
The experimental methods of the classification process can be seen in Fig.3. In this case, an experiment is conducted by measuring several metrics, mainly accuracy and computing performance to find the best classifier. There are four classification algorithms tested in this experiment: Decision Tree, SVM, Random Forest, and Adaboost. The best parameter is searched for each classifier with the process of hyper-parameter optimization. The parameters and their data type are shown in Table 5 as follows. The process of hyperparameter optimization is shown in Fig.4. Each classifier is optimized in 1000 epochs using three optimizer algorithms: Forest, GBRT, and Gaussian Process. Accuracy in each epoch is measured using Cross-Validation with 3 folds, which then averaged to be used as the metric of the optimization. The average accuracy is multiplied with -1 as the optimizer set as a minimizer of the value function. Parameters of the best result in each classifier are then used as the parameters to train the final classifier.
The final training process of the classifier can be seen in Fig.5. Each classifier is trained for the final model test using the best configuration of the parameter. After training, the model is tested using a test dataset and then we calculated the performance of each classifier using several metrics measurements to obtain the best performance of the classifiers.
There are several metrics measured in this final model test. The first one is accuracy. Accuracy simply measures the percentage of correctly identified conditions. Accuracy is denoted as in (8) where is true positive, being true negative, is false positive, and being false negative. The next metrics are Precision and Recall. Precision is a fraction of true predicted instances over all of the predicted instances. While Recall is a fraction of true predicted instances over all of the true instances. Precision and Recall are calculated as in (9) and (10).
Precision and recall are often in the inverse relation, where trade-offs cannot be avoided to choose one over the other. In some cases, precision is better to measure than recall and vice versa. But in the more general case, a harmonic average of precision and recall can be utilized as a single metric to measure the model. This harmonic average of precision and recall is called F1-score or F-measure and can be calculated as in (11).

Results and Discussion
The methods that have been proposed in the previous section are then implemented on the system. First, we optimized the hyperparameters of each classifier. The second one is we noted the configuration result of the hyperparameter from the training process and tested the hyperparameter configuration of each classifier to analyze the Copyright © 2021 MECS I.J. Intelligent Systems and Applications, 2021, 5, 1-13 performance. The results are discussed in the following sub-section.

Hyperparameter Optimization
Classifier algorithms have several parameters to set. The parameters need to be optimized for classifiers to reach the global minimum of error as close as possible. There are 12 configurations of optimization tested in this experiment. Each configuration is optimized in 1000 epoch. The convergence of the optimization showed in Fig.6. Fig.6. The convergence of the optimization As can be seen, most classifiers need less than 200 epochs to converge, with only minor improvement seen after 200 epochs. Most classifiers also have relatively significant improvement within 100 epochs of optimization, with the exception of SVM. Adaboosts has around 5% improvement, Decision Trees has around 17% improvement, and Random Forest has around 24% improvement. Interestingly, while SVM has the lowest improvement, at 0 epoch SVM is still better than all of the other classifiers after convergence. The result of each configuration after 1000 epochs can be seen in Table 6. The best configuration of each classifier is noted, then the parameters of each configuration are applied to the final model training.

Final Model Test
Each classifier algorithm is then trained for the final model test, using the parameters of the best configuration of each classifier. After training, the model is tested using a test dataset with the result that can be seen in Table 7  As shown in Table 3, SVM scores the best in all of the measurement metrics compared to all of the other classifiers. Random Forest closely follows SVM, especially in the Precision score with the difference of ±0.0003 points. The Adaboost classifier scores the worst with a relatively significant margin. While the other classifiers scores above 94% of accuracy, Adaboost falls behind with 77% accuracy. The confusion matrix of the SVM test can be seen in Fig.7. These results are also confirmed by research that has been done in the last five years that used SVM as a classifier for face recognition and has been proven in making a high level of accuracy. This SVM classifier algorithm is used for face recognition research with various purposes, such as P. C. Vasanth and K. R. Nataraj [38] used SVM for expression classification and produced good accuracy in displaying the expression and facial action units. Rustam and Ruvita [39] also used the SVM classifier for face recognition systems to perform gender classification by comparing the polynomial kernel (d = 5) and RBF kernel (σ = 0.05) the and the result is SVM with a polynomial kernel (d = 5) reached 100% of accuracy. Rayani and K. Rajakumar [40] obtained an accuracy of 85% while using SVM for face detection and face recognition. Another one is Sharma and Sachdeva [41] who obtained an accuracy value of 98.7% by utilizing SVM as a classifier for face recognition. Prakash and Singh [42] reviewed numerous papers and concluded that SVM has a more effective technique than the others and SVM can be trained successfully for face recognition and SVM is also a better learning algorithm compare to the nearest center approach in terms of face recognition, And the last, P. S. Hiremath et al. [43] and A. Tofighi et al. [44] have been comparing the SVM and K-Nearest Neighbor (KNN) for face recognition and got the result that SVM performs better than KNN with the scores are 99.2% on 3D face recognition [43] and 93.5% on recognizing the face of an image and a video [44].

Conclusions
An analysis has been made for the performances of four different classifiers which are SVM, decision tree, random forest, and Adaboost. This is done by calculating some metrics including recall, F1-score, accuracy, and precision. From the experiment, we obtain that SVM gives the best performance in all measurement matrices and reached an accuracy score of 98%. Hyperparameter tuning is proven to improve the performance of a classifier, especially in terms of accuracy, for example, SVM. In previous studies, it showed that SVM is not the best classifier compare to other classifiers, but when we optimize the hyperparameter like in this experiment, SVM has the highest accuracy than others. It proves that hyperparameters are critical in building a robust and accurate model of the classifier. This study also enriches a reference to the comparison of the classifier algorithm that developers/programmers can use in choosing the best classifier algorithm according to their needs. Further, this research is expected to continue by modeling the relationship between each hyperparameter, hence, it will be easier to tune and obtain the best classifier as needed. Another, we can also make another research to obtain a high accuracy value classifier that also has the best performance in a real-time embedded system with less time-consuming, efficient, and cost-effectively, due to face recognition creates a potential for many businesses aspect.