{'fraction_non_alphanumeric': 0.03358528237307473, 'fraction_numerical': 0.029378208784940102, 'mean_word_length': 4.7408923454768725, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Performance evaluation of naive Bayes and support vector machine in type 2 Diabetes Mellitus gene expression microarray data

Type 2 Diabetes Mellitus (T2DM) is a metabolic disorder that the number of diabetics increases every year. So that prevention is needed by knowing the trigger of T2DM. Gene expression microarray data contains information of gene that can be used to determine the causes of T2DM. It is necessary to use certain techniques to analyze gene expression microarray data because it has a large amount of data and attributes. This study aims to evaluate the performance of algorithms in classifying gene expression microarray data. Algorithms that were used in this study were Naive Bayes, and Support Vector Machine (SVM). SVM used many kernels function such as Linear, Radial Basis Function (RBF), Polynomial, and Sigmoid. Information gain was used to select the features in GSE18732 dataset by choosing top 10, 20, 30, 40, and 50 features. Performance of algorithms was evaluated and compared by using 30% testing set and 20% testing set. The results of the study indicated that SVM using Polynomial kernel had a high performance if it was compared to other algorithms. It achieved 98.15% accuracy using 30% testing set and achieved 100% accuracy using 20% testing set.


Introduction
Type 2 Diabetes Mellitus (T2DM) is a metabolic disorder caused by the pancreas not producing enough insulin or the body cannot use the insulin that is produced effectively so that there is an increase in the concentration of glucose in the blood [1]. In 2013, the International Diabetes Federation (IDF) estimated the number of diabetics in the world to be 382 million and estimated that diabetics in 2035 would continue to increase to 592 million people [2].
Microarray technology is a technology in biomedicine that can be used to assess gene expression, genomic structure analysis, identification of genetic polymorphisms or detection of viruses, bacteria and pathogenic fungi [3]. So that gene expression data from these technologies can be used to analyze genes that affect T2DM disease and classify T2DM samples with normal samples. However, analyzing datasets from microarray gene expression data is a challenge in data mining techniques because the data is very large and consists of thousands of attributes [4]. It is necessary to use the right method to analyze very large data.
Research on disease classification using microarray gene expression data has been done before. The study was conducted by identifying T2DM disease in microarray gene expression data using the Performance evaluation of naive Bayes and support vector machine in type 2 Diabetes Mellitus gene expression microarray data Ramdaniah 1* , A Lawi 2* and S Syarif 1* Biomarker Module (32 genes) with an accuracy of 84.79% and comparing it with the Support Vector Machine (SVM) method combined with Recursive Feature Elimination (RFE) with an accuracy of 73. 24% [5]. Al-Sabti et.al [6] classified T2DM in several gene expression data, one of them were GSE18732 data. The results showed that the accuracy of the classification using Identified Biomarkers achieved 84.71%, Module Biomarker (32 genes) achieved 92.39%, SVM-RFE achieved 67.39%, and Pathway Activity inference using Condition-responsive genes (PAC) achieved 84.78%. Kourou et. al [7] classified oral cancer in gene expression data using several methods. The results showed that the classification accuracy using the Naïve Bayes method achieved 70.80%, Bayesian Network (BN) achieved 83. 20%, SVM achieved 78.40%, Artificial Neural Network (ANN) achieved 83.90%, Adaboost achieved 82.5%, and the Random Forest achieved 89.30 %. Widiawati et. al [8] classified specific organ expression in maize using gene expression data. The researcher classified the data using K-Nearest Neighbor (KNN). The results showed that the accuracy using the proposed classifier achieved 93.33%.
Previous studies that related to the classification of gene expression data have not yet achieved high performance, so that we proposed and evaluated the system that can classify Type 2 Diabetes Mellitus in gene expression data using several classifiers such as Naïve Bayes, and Support Vector Machine (SVM) and use Information Gain (IG) to select features in gene expression microarray data to achieve high performance in classifying data.

Materials and Methods
Several stages and algorithms are needed in classifying GSE18732 data. Based on Figure 1, the data that used was GSE18732 gene expression data and then the data was processed at the pre-processing stage. Data transformation was carried out during the pre-processing stage using logarithmic transformations. Furthermore, the data were normalized using quantile normalization. In the feature selection stage, the algorithm that used was Information Gain. Then the data entered the resampling stage using the SMOTE algorithm.
The method that used in classification were Naïve Bayes, and SVM using Linear, RBF, Polynomial, and Sigmoid kernels. Furthermore, the performance of the algorithms was evaluated by calculating the value of accuracy, sensitivity, and specificity. The stages carried out in the study are as follows:

Dataset
The data that used is publicly accessible data provided by the National Center for Biotechnology Information (NCBI) [9]. Data can be accessed at https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE18732. The input data that used consisted of 118 samples and 25770 features.

Logarithmic Transformation
The method used in the pre-processing stage was Logarithmic transformation. Logarithmic transformation was used to change the scale of measurement of original data into other forms so that data can fulfill the assumptions underlying the various analysis. Logarithmic transformation (usually base 2) is often applied to microarray data. Equation (1) is a logarithmic transformation based on previous research [10].
Where the original data is and ′ is the data from the logarithmic transformation.

Quantile Normalization
Quantile normalization method was used in the pre-processing stage after processing data in transformation. The purpose of the quantile method is to make the distribution of the probe intensity for each array in the same set of arrays. This method is derived from the idea that quantile-quantile plots show that the distribution of two data vectors is the same if the plot is a straight diagonal line and is not the same if in addition to the diagonal line [11]. Equation (2) is a formula for quantile normalization. Given = ( 1 ,…, ) for = 1, … , is a vector from quantile th for all array

Feature Selection
In the feature selection stage, the algorithm used was Information Gain. Information Gain is a method for selecting features by ranking attributes. The formula for calculating entropy can be used using equations (3) and (4) based on the explanation in the study [12].
Where ( ) represent probability function from . Equation (5) is a formula for calculating information gain. Information Gain value is obtained from Entropy(X) minus Entropy(X|Y).

Synthetic Minority Over-sampling Technique (SMOTE)
SMOTE is used to solve classification problems. SMOTE is useful for balancing datasets to improve the learning algorithm that will be used. Equation (6) is used to added new sample into dataset [13].
Where = 1, 2, … , N, rand represents random number between 0 and 1. represents new sample. represents sample and [ ] representing a sample of neighbors th.

Naïve Bayes
Naïve Bayes is a classification method that predicts future opportunities based on previous experience. Naïve Bayes classifier is assumed that certain characteristics in the class does not have relation with other class [14]. Equation (7) is a formula of Naïve Bayes [15].

Support Vector Machine (SVM)
SVM aims to find the best classification function to distinguish examples from two classes in training data. In the SVM there is a hyperplane ( ) in the middle of the class and separates two classes. After the f (x) function is found, new data items can be classified by checking the function sign ( ). will be in a positive class if ( )>0. Equation (8) is used to calculate the maximum margin of hyperplane [16] Where is the number of training samples, where = 1 … is a positive number, is multipliers Lagrange and is Lagrangia, is constant and identify hyperplane. The kernel functions that used are Linear kernel, Radial Basis Function (RBF), Polynomial, and Sigmoid. The following equation is kernel used.
( , ) = tanh ( + ) (12) Equation (9) is a linear kernel that has the simplest function. Equation (10) is RBF kernel that is parameter that can be adjusted and used to calculate the performance of classifier. Equation (11) is the Polynomial kernel where and are feature vectors, is parameter that can be adjusted, is constant, is degree of Polynomial. Equation (12) is Sigmoid kernel where and are parameter that can be adjusted.

Performance Evaluation
The confusion matrix is one way to analyze the performance of the classifier used. The confusion matrix is a two-dimensional matrix consisting of the actual class of an object and class as a result of classification. Confusion matrix often uses positive and negative [17]. Calculating accuracy, sensitivity, and specificity are needed to evaluate performance of the algorithms. Equation (13) is the formula to calculate accuracy, while equation (14) is used to calculate sensitivity and equation (15)

Feature Selection
The feature selection was done after the data was processed in the pre-processing stage. Information gain was used to select the best features of 25770 features. Then the 50 best features were selected based on the highest Information Gain value. Then the dataset was divided into 5 types, namely dataset1 which consists of 10 best features, dataset2 consists of 20 best features, dataset3 consists of 30 best features, dataset4 consists of 40 best features and dataset5 consists of 50 best features.

Resampling using SMOTE
The GSE18732 data consisted of 118 rows and had two classes, namely Type 2 Diabetes Mellitus and normal class. But the amount of data in both classes is different. Data on T2DM class consisted of 46 rows while data in normal class consisted of 72 rows. It can affect the performance of the classifier so it was necessary to use the SMOTE algorithm so that the amount of data in both classes was balanced. The number of rows in each class after using the Smote algorithm was 92 rows. So that the total row of data after using the SMOTE algorithm was 184 rows.

Performance Evaluation and Discussion
Data was divided into two types of testing sets, namely data with 30% testing sets and data with 20% testing sets before classifying data. Both types of testing sets were used to compare the performance of the classifier that was used. Table 1 shows the performance of Naïve Bayes. Based on table 1, the highest accuracy value using Naïve Bayes is 88.89% by classifying dataset1 which consists of 10 best features.  Table 2 shows the SVM performance using a Linear kernel with parameter cost = 10. The classification results using the Linear kernel achieved an accuracy of 92.59%, a sensitivity of 96.30% and a specificity of 88.89% in classifying datasets with 50 features.  Table 3 shows SVM performance using RBF kernels with cost = 1 and gamma = 1. Based on table 3, the highest accuracy achieved 94.44%, sensitivity achieved 88.89, and specificity achieved 100%.  Table 4 shows the SVM performance using the Polynomial kernel with parameters cost = 10, gamma = 0.1, and degrees = 2. Table 4 shows that the highest accuracy using the Polynomial SVM kernel achieved 98.15% in classifying datasets with 30 features.  Table 5 shows the performance of SVM with the Sigmoid kernel using parameters cost = 1 and gamma = 0.1. The highest accuracy using the SVM Sigmoid kernel was 83.33% in classifying dataset1 which consists of 10 best features.  Figure 3 shows a comparison of the algorithms used in the dataset consisting of 70% training sets and 30% testing sets. The classification results show that SVM polynomial kernel produces the highest accuracy of 98.15%.      Figure 5. Comparison of Classifier Specificity using 30% Testing Set

Algorithms Performance using 20% Testing Set
The following is the performance of the algorithms that were used in classifying data using 20% testing sets. Table 6 shows the performance of Naïve Bayes on data with 20% testing sets.  Table 6 shows that the highest accuracy using Naïve Bayes on data with 20% testing set is 91.67%. Table 7 shows the classification results using the SVM Linear kernel. The highest accuracy using Linear kernel SVM is 94.44% with parameter cost = 10.  Table 8 shows SVM RBF kernels performance has accuracy 97.22% using cost = 10 and gamma = 0.1 parameters by classifying dataset that consists of 10 features.  Table 9 shows performance of SVM Polynomial kernel accuracy using cost = 10, gamma = 0.1, and degree = 2. It achieves 100% accuracy in classifying dataset with 40 features.    Table 10 shows the performance of SVM Sigmoid kernel achieves 94.44% accuracy using cost = 1 and gamma = 0.1 parameters.  Figure 6 shows the difference in classifier accuracy. Based on Figure 7, SVM using the Polynomial kernel has the highest accuracy in classifying data with 20% Testing Sets. Polynomial kernel SVM achieves 100% accuracy, 100% sensitivity, and 100% specificity in classifying datasets consisting of 40 best features.

Discussion
Performance evaluation of algorithms in classifying gene expression microarray data uses 30% testing set and 20% testing set. Table 11 shows the difference in classifier accuracy using different testing sets. Based on table 11, the performance of the algorithm was increased when using 20% testing sets. SVM with a polynomial kernel has the highest performance compared to other algorithms. Polynomial SVM achieves 98.15% accuracy using 30% testing set and 100% accuracy using 20% testing set.