{'fraction_non_alphanumeric': 0.025366876310272537, 'fraction_numerical': 0.013888888888888888, 'mean_word_length': 4.911090458488228, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Comparison of Machine Learning Algorithms for Autism Spectrum Disorder Classification

Autism Spectrum Disorder is one of the fastest-growing neurodevelopmental disorders in the world. These neurodevelopmental disorders often attack children, affecting social development and behavior. Effective early detection of ASD is needed to reduce the risk of ASD in children. This study classified the ASD using KNN, SVM, Random Forest, Backpropagation, and Deep Learning. The Children ASD dataset consists of 292 subjects, 49 ASD subjects and 243 normal subjects. This adolescents ASD dataset consisting of 104 subjects with 63 ASD subjects and 41 normal subjects. The classification process is done by applying some parameters to each algorithm. The algorithms performance in classifying ASD dataset was compared. Based on the specificity and sensitivity value of the Random Forest algorithm with full features is the best algorithm compared to other algorithms in classifying ASD in children and adolescents.


INTRODUCTION
Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that can affect the development of social and communication behavior, stereotypical behavior, and interest in a matter [1]. Etiopathogenesis of ASD is still unknown with certainty, but several studies have argued that ASD is generally influenced by several biological factors such as genetic defects, inflammation of the brain, and abnormal conditions during pregnancy [2]. Besides that, genetic and environmental interactions such as autoimmunity can also be a factor causing ASD [3].
There are around 1.47% of the world's child population experiencing ASD [4]. Based on the National Health Interview Survey in 2014, ASD cases experienced a significant increase with an estimated prevalence of 1:45 and an increase in ASD cases was most prevalent in children [5]. The prevalence of ASD cases in children in the United States reaches 1:59, the prevalence is almost one-third of ASD cases that occur in children in the world [4]. According to the World Health Organization (WHO), an estimated 1 in 160 children in the world has ASD [6]. The prevalence of autism according to ASA (Autism Society of America) data in 2000 is 60 per 10.000 births, with a total of 1: 250 populations [7].
Early detection of ASD in children needs to be done to ensure that children with ASD get appropriate support from the surrounding environment [8]. Besides, early detection of ASD can also speed up health services, to reduce the burden of ASD sufferers and their environment [9]. To get ASD detection results reliable can be done in the age range of 18 to 24 months [10]. Some methods that can be used in the detection of ASD in children can be through consideration of family medical history or with biomarkers in children [11].
Several studies have been conducted as an effort to detect early cases of ASD, especially in children. Screening ASD has been done by Charman, by comparing Social Communication Questionnaire (SCQ), Social Responsiveness Scale (SRS), and Children's Communication Checklist (CCC) methods. The SCQ method has a sensitivity of 0.86 and a specification of 0.76. While the SRS method has a sensitivity of 0.78 and a specification of 0.67. And the CCC method has specifications of 0.93 and specifications 0.46. Three screening methods can identify ASD in school-age children [12].
Kang's studies identifying cases of ASD in children using Support Vector Machine (SVM) based on EEG and eye-tracking data [13]. The classification results using EEG and eye-tracking data can be improved then only used EEG or eye-tracking. Research on the classification of epilepsy and ASD using feature selection DWT and cross-correlation as well as methods ANN, KNN, SVM, and LDA classification based on EEG was conducted by Ibrahim [14]. The combination of feature selection DWT and Shannon Entropy method with the KNN classification method is the best in EEG classification of three classes, normal, epilepsy, and autism.
Other studies conducted by Hadoush in identifying the severity of ASD in children using empirical mode decomposition and second-order difference plot [15]. Data features of different IMFs, SODP plot patterns, elliptical areas, and CTM values indicate children with low and severe ASD levels. Based on the ANN classification model, the EMD value measurement results can be used as an automatic tool that can distinguish the severity of ASD children.
Some studies only screen ASD manually. Some studies have classified using machine learning but not many use the Random Forest and Deep Learning methods. Besides, there is still no ASD classification based on DSM-5 results comparing the performance of several methods of machine learning. So that in this study using the KNN, SVM, Random Forest, Deep Learning, and Backpropagation methods and compare the performance of these methods in the classification of ASD in children based on the results of DSM-5.

K-Nearest Neighbors (KNN)
K-Nearest Neighbors (KNN) is the simplest algorithm in the artificial learning algorithm [16]. To classify existing samples, the closest sample is chosen based on a certain distance. The new sample class is determined through the closest neighbor k class which often appears. The selection of k is done a priori with cross-validation or heuristics. In binary classification, k chosen is an odd number to avoid the similarity of the number of classes of the nearest closest neighbor [17]. To calculate the closest distance can be done with the distance Euclidean, City-block, and Chebychev. (1)

Support Vector Machine (SVM)
Support Vector Machine (SVM) is one of the algorithms in machine learning that is often used in classifications in various fields [18]. SVM looks for an optimal hyperplane as a separator of two classes in the process classification. An optimal hyperplane is obtained from optimal margins. Margin value is the distance between classes. Support vectors are the points in each class that are the closest to the hyperplane.
Suppose there are two classes in the dataset namely and by hyperplane , the decision rule is defined as : (2) Then find the minimum value of: (3) to find the optimal hyperplane with the following conditions: Next, to determine the data class, the decision function is calculated (5) where m is a lot of support vectors, is the weight of each data, and is a kernel function [19].

Random Forest
Random Forest is a combination of classification and regression trained on a dataset that has the same size as the training set and called bootstraps. Bootstraps were obtained from a random sample selection on the training set [20]. In the classification process, each tree represents one class, and the forest predicts which class has the votes most. The training algorithm in random forest uses aggregating bootstraps or bagging techniques. Suppose training set and response . Bagging repeated predictions sample select a random sample by replacing the training set and is obtained by the average prediction of all regression tree at is: (6) or with take the majority of classes on trees.

Deep Learning
Deep Learning is one method of machine learning that has a lot of attention [21]. Deep learning can predict models with high complexity and can be applied to various applications [22]. In this study, the instance iterator used is the default instance iterator with a simple ARFF file without spatial interpretation. The network configuration used is the neural network configuration and the iterator listener settings used are the epoch listener where this listener is used to evaluate the model when training for every N epochs. In this study, the CNN model used is CustomNet.

Backpropagation
Backpropagation is an algorithm that adds delta rules to Multi Layers Perceptron (MLP) as a backward phase [23]. In this study, Backpropagation is abbreviated as BP. Backpropagation is a systematic method for multilayer Artificial Neural Network (ANN) training because backpropagation has three layers in its training process, namely the input layer, hidden layer, and output layer. The backpropagation algorithm reduces the error rate by adjusting the weight based on differences in output and desired targets.
In this study, experiments were carried out using WEKA version 3.8.4. Waikato Environment for Knowledge Analysis (WEKA) is a computer program developed to identify information from raw data. WEKA develops machine learning and data mining by applying algorithms to data [24]. K-folds cross-validation is dividing the sample into k samples of the same size, each subsample is taken as validation data to test classification and repeated k times [25]. The inputted data is then cross-validated with 10-fold cross-validation. Data classification is done by four methods, namely K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Random Forest, Deep Learning, and Backpropagation. Some parameters are changed in each classification method to determine the effect of parameters on the results of data classification.

Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5)
Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition is a diagnostic and taxonomic tool that is updated in 2013 and published by the American Psychiatric Association (APA). DSM is used as the main alloy for psychiatric diagnoses in the United States [1]. In DSM-5, there are concise and explicit criteria to facilitate an objective assessment of symptom presentations.

RESULTS AND DISCUSSION
In this study, there are two datasets, they are child and adolescent data. The dataset that is used in this study comes from the UCI Machine Learning. UCI Machine Learning provides 507 datasets that can be used by the machine learning community in developing research [26]. ASD and normal data were obtained based on observations and weighting of ten features of children and adolescent behavior with ten individual characteristics that were effective in detecting the ASD cases in children and adolescents from controls in behavior science. There are 21 attributes in the dataset. Child data are taken from 292 subjects consisted of 49 ASD subjects and 243 normal subjects. The age range of subjects is 4 to 11 years, with a total of 208 male subjects and 84 female subjects. Adolescents' data are taken from 104 subjects consisted of 63 ASD subjects and 41 normal subjects. The age range of subjects is 12 to 16 years, with a total of 50 male subjects and 54 female subjects [26] [27].
The machine-learning algorithm that has been used for ASD classification are KNN, SVM, Random Forest, Deep Learning, and Backpropagation. All algorithm has been run with various parameters. The evaluation measure that is used in this classification is accuracy, precision, recall, ROC, specificity, and sensitivity. Accuracy in classification is the percentage of accuracy of records data that are classified correctly after testing the results of classification [28]. The accuracy value is formulated as (7). Besides accuracy, in classifying data need to consider the level of precision, recall, ROC, specificity, and sensitivity in the classification process. Precision is the level of accuracy between the desired information and the results of predictions made by the system formulated as (8).
The recall is the level of success of a system in classifying data correctly which is formulated as (9). Receiver Operating Characteristic (ROC) is a measure of the performance of a classification problem in determining the threshold of a model. The ROC curve is formed based on the True Positive Rate and the False Positive Rate. The higher the True Positive Rate and the smaller the False Positive Rate, resulting in a better threshold. Specificity is the level of accuracy in Advances in Engineering Research,volume 196 predicting data in a negative class, formulated as (10). Sensitivity is the level of accuracy in predicting data in a positive class, formulated as (11) (9) (10) Based on Figure 3, the highest accuracy is achieved in the ASD classification process in children using the Random Forest algorithm with full features which is 1. Then followed by Backpropagation and SVM algorithm with C=5 and ε= at 0.9965. By using the Deep Learning algorithm with E=15 the accuracy is 0.9589 and the KNN algorithm with K=5 is 0.8973. The higher the accuracy value generated by the system in the classification process, the more precise the system is in predicting all data. Based on Figure 4, the highest accuracy is achieved in the ASD classification process in adolescents using the Random Forest algorithm with full features which is 1. Then followed by the KNN algorithm with K=1 is 0.9038. By using Backpropagation and SVM algorithm with C=5 and ε= at 0.8942. And the Deep Learning algorithm with E=15 the accuracy is 0.8846. The higher the accuracy value generated by the system in the classification process, the more precise the system is in predicting all data.
In addition to using accuracy in data classification, a value is needed kappa statistical to strengthen the confidence of the truth of the classification results. Kappa Statistic is a statistic that has a range of values from 0 to 1 and is used to test interrater reliability. The higher the kappa statistics, the better the algorithm used [29]. In this study, the Random Forest algorithm has the highest statistical kappa value, which is 1. Figure 4 The highest accuracy of each algorithm in the classification process in adolescents The study was conducted by applying various parameters in each algorithm. The best results classification from each algorithm is seen in Table 1. Based on Table 1, the Random Forest algorithm with full features has the highest precision value with a value of 1, this means that the algorithm Random Forest with full features has a high level of accuracy between the desired information and the predicted results provided by the system, then followed by other algorithms. Random Forest algorithm with full features also has the highest recall value which is 1 compared to other algorithms which means that the Random Forest algorithm with full features also has a very good level of accuracy in classifying data correctly. The threshold generated by the Random Forest algorithm with full features is also the best, this is indicated by the highest ROC value found in the Random Forest algorithm with full features so that it produces the best classification result compared to the Backpropagation, SVM, Deep Learning and KNN algorithms.
In predicting data on negative classes (ASD), the Random Forest algorithm with full features, SVM with C=5 and ε= and Backpropagation with 10 hidden layers has a very good level of accuracy with a value of specificity 1. But in predicting data in a positive class (normal), the Random Forest algorithm with full features has the highest level of accuracy with a value of sensitivity 1 followed by the Backpropagation, SVM algorithm, Deep Learning, and KNN. The greater the specificity and sensitivity of the algorithm, the level of accuracy in predicting data is also getting better.
In the ASD classification process in children and adolescents, each algorithm with various parameters is calculated training time and testing time. Training time is the time needed to build a model in the classification process. While testing time is the time needed to test data in the classification process. The best training time and testing time for each algorithm are shown in Figure 5 and Figure 6.  Based on Figure 5, the time needed for KNN, SVM, Random Forest, Deep Learning, and Backpropagation algorithms for training ASD data is quite short. The training time data on the Deep Learning algorithm is greater than other algorithms. This is because the deep learning algorithm is a complex model with a large number of parameters that require more time in execution. While the lowest training time is on the KNN algorithm because KNN is one of the simplest methods in data classification. KNN algorithm is faster for Advances in Engineering Research, volume 196 training because it does not require a learning process, there is no process of making a model. The existing dataset is only stored. When there is incoming test data, the class will look for the closest similarity to the test data.
Based on Figure 6, the time needed for the Deep Learning algorithm is greater than other algorithms. While the lowest testing time is on the Random Forest and SVM algorithm. The ASD Screening dataset for children has been used in research publications by Fadi Thabtah [27]. Their research publications explained that the classification of ASD by using machine learning but did not review about machine learning and its accuracy in the classification process. Thabtah only discusses the dataset and DSM-5 [27]. This study wants to complete Thabtah studies with the contribution to find the best machine learning algorithm for ASD classification.  The previous study with the title a comparison of machine learning algorithms for surveillance of autism spectrum disorder compared the performance of the algorithms across 10 random train-test splits of the data, using classification accuracy, F1 Score, and the number of positive calls to evaluate their potential use for surveillance [30]. This study wants to compare the performance of the algorithms in machine learning by their accuracy, kappa statistic, training, and testing time in the classification of ASD.

CONCLUSION
A previous study with the title ASD data classification in children and adolescents by machine learning but did not review machine learning along with accuracy, only reviewed the dataset and DSM-5. This study compares machine learning in ASD classification with a focus on the performance of the algorithms, with some evaluation measurements are the value of accuracy, kappa statistic, classification time, and other aspects. The best algorithm in ASD classification in children and adolescents is obtained in this study.
In this experiment, based on the accuracy, specification, and sensitivity values for ASD classification, the Random Forest algorithm has the highest performance than the other algorithms. The value of kappa statistics on the Random Forest algorithm is 1, this means that the results of the ASD classification using the Random Forest algorithm have a high level of true confidence. Thus, the Random Forest algorithm is the best in the classification ASD compared by other algorithms. KNN algorithm shows good training and testing time because this algorithm does not need time to build the model or learning phase, and this dataset is very simple so simple algorithm able to handle it. Deep Learning algorithm is no better than the Random Forest algorithm because the dataset used is simple namely 21 features. Deep Learning is intended for datasets with a very large number of features so that if used for data with simple features, it causes high computational costs. So, for this problem, simple algorithms are better performing.