{'fraction_non_alphanumeric': 0.030322073841319717, 'fraction_numerical': 0.01673212882953653, 'mean_word_length': 5.022232734153264, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Comparison of Machine Learning Model to Predict Bankruptcy: Indonesian Stock Exchange Data

This study aims to determine the Machine Learning Model used to predict bankruptcy. The data was conducted from the financial statements of two public companies reported by the Indonesia Stock Exchange from 2009 to 2015. This research method uses an analysis feature in which the accounting ratios are used in statistical analysis of financial statements that handle missing values, choose the correlation feature related to class, and dealing with unbalanced datasets. This problem was resolved at the beginning of the pre-processing phase. The training process uses pre-processing results to fit the data with the prediction model. Accuracy is used to measure the performance of the model in predicting bankruptcy. The result is Sequential Minimal Optimization (SMO) with linear kernel function that works best to predict 1 year before bankruptcy with an accuracy of 91.57% and SMO with Radial Basis Function (RBF) works well to predict 2 years before bankruptcy; the accuracy is 93.8%. This study shows the effect of feature selection and normalization process in making correct predictions using the SMO method.


Introduction
The bankruptcy of a company is a condition where a company is unable to repay a debt that has matured, or a company declared bankrupt by the court. It is important for investors, creditors or financial institutions to analyze conditions of the public company. The result helps them to determine the policies. For this reason, every year issuers must submit their financial reports to the public. Based on the financial statements, it calculates the financial ratios and those ratios are analyzed using statistical analysis. Business conditions becoming more complex, dynamic and difficult to fulfill the assumptions of normal, homogenous and independent. Therefore, Machine Learning can be required to predict bankruptcy.
According to the prediction of bankruptcy by Altman [1], he used the Multivariate Discriminant Analysis (MDA) method to determine the company's bankruptcy. The problems arise when using MDA are a prerequisite for normality of data, inequality of dispersion matrices from all groups and unfulfilled non-random-sampling requirements of bankrupt or non-bankrupt companies [2]. Barboza et al. used machine learning to predict corporate bankruptcy [3], they used 133 bankrupt companies and 13300 solvent company data. The results showed that the Random Forest method works properly between Linear Discriminant Analysis, Logistic Regression, Neural Network, Support Vector Machine, Boosting, and Bagging. Different results were shown by Klepac and Gogas [4][5] which found that SVM using RBF function worked well for predicting bankruptcy. Klepac used data from retail companies while Gogas used SVM to predict bankruptcy at the Bank. Antunes [6] indicated that the Gaussian Process worked better than the SVM or Logistic Regression to the data from the French market. A comparative study carried out by Tsai [7] to compare several methods in machine learning to predict bankruptcy. The difference in stock exchange data characteristics of each country lead to  [8] research explained the need to pay attention to the imbalanced data set conditions that exist in data of bankrupt companies. Le [9] described the oversampling technique to overcome unbalanced data problems. The existence of a missing value problem is something that can solved in preprocessing to ensure the success of predictions using machine-learning methods. The needed for financial ratios selection also influences the bankruptcy model. Wang, Liang, Dellepiane [10][11][12] have shown the effect of using feature selection on prediction results.
This study aims to determine the Machine Learning Model used to predict bankruptcy. Therefore, in this study we will examine several supervised learning methods to get the best model that can predict one before and two previous bankruptcies. The model considers various problems such as unbalanced datasets, selection of appropriate financial ratios, and handling incomplete datasets. In addition, this study will also determine ratios that affect the determination of company bankruptcy based on Indonesian stock exchange data. The stages carried out in this research are preprocessing, feature selection, training, and testing.

Method
The experimental method used to get a predictive model and suitable machine learning methods. Following are the steps of the research carried out:

Collecting data
The processing data are financial reports taken from public companies of the Indonesian Stock Exchange from 2008 -2014. It consists of 100 solvent companies and 20 bankrupt companies. Table 1 is 21 parameters used to predict the condition of the company. The parameters used are the value of financial ratios. Financial ratios are commonly used by accountants to evaluate the condition of the company based on audited financial statements [3,13].

Analysis of Process
Data processed through the preprocessing stage. The following problems considered in preprocessing are the appearance of the missing value data on the observed feature values, the need to normalize and the need to standardize the data. Training and testing are carried out to get a prediction model. It uses the 5-fold cross-validation to test the model. The method used is Baseline Classification (ZeroR), Naïve Bayes (NB), Support Vector Machine (SVM) using a linear kernel, Sequential Minimal Optimization (SMO), Logistic Regression (Logistic), K-nearest Neighbor (K-NN), Decision Tree (C45). It is giving a greater weighting to the cross-validation table for classes with fewer data to handle unbalanced data.

Learning and Testing
After obtaining the model and selecting the method, then the model accuracy will be tested. The data are splitting 60% of training and 40% of testing data. The testing measures the performance of the model.

Discussion
In this section, it is compared the research with the others. Furthermore, it is analyzed the features chosen from the selection results by comparing it with the theory of financial ratios commonly used in determining bankruptcy.

Results and Discussion
Weka 3.8 application used to measure the performance of the model. Meanwhile, Brownly [14] explains the steps of using Weka especially for binary classification on his website. Model testing uses all data and accuracy testing uses 5-fold cross-validation.  The accuracy of 83.33%, the Baseline Classification method (ZeroR), is a reference to describe the other methods worked in classification [14]. Table 2, there are four conditions tested, process data without preprocessing, normalized data, standardized data, and processing of missing value data. Each model examined to predict bankruptcy one year and two years before. Naïve Bayes got the highest accuracy, 83.17%, in the 1-year before bankruptcy prediction and 80.33%, in the 2-years before bankruptcy prediction. This result corresponds to the theory which states Naïve Bayes requires the assumption that each feature must follow a standard normal distribution [15]. SVM using Radial Basis Function (RBF) as a kernel function, gamma = 1 and Sequential Minimal Optimization using Linear Kernel Functions show consistency in predicting bankruptcy with an accuracy value above 80%. Multiple Logistic Regression (Logistic) and Decision Tree (C45) produces an accuracy value that is no better than the ZeroR method. Using neighbors K = 3, The K-NN method shows good results only in predicting bankruptcy 2 years prior.
Based on Table 2, the results of the two-tails statistical test using the t-distribution that compares each method. The SMO method shows that the Logistics Method and C45 produce significant differences with SMO. The statistical hypothesis results, in Table 2, show that the accuracy of C45 (77.00%) is different significantly with SMO (87.00%) in predicting bankruptcy 2-years prior. Furthermore, for the feature selection process, it uses Chi-Squared, Pearson Correlation (PC), Gain Ratio (GR), and Information Gain (IG) methods. All data through the normalization process. Each feature ranked based on the calculated value of Chi-Squared, PC, GR, and IG. Seven of the highest IOP Conf. Series: Materials Science and Engineering 662 (2019) 052019 IOP Publishing doi:10.1088/1757-899X/662/5/052019 4 value features of each selection method are selected as features and will be used to obtain prediction models. Especially for the Pearson Correlation, all tested variables have a correlation value. Therefore, it was selected 7 of the variables with the highest values as features for the Pearson Correlation. Table  3 shows the details of the results. Although the ranking features in Chi-Squared, GR and IG are not the same, the seven features that appear are the same, except for the Pearson Correlation. Gross Profit Margin feature appears as a relevant feature.
Based on Havier and Qiyu's research [16][17], ROA always appears as a feature to predict bankruptcy. Operating Profit Margin, Equity to Asset, Net Profit Margin are features that are also used by Havier, Qiyu and Tian's research [16][17][18]. From table 3, it concludes that feature selection using Pearson Correlation is the best choice to use as a feature selection method. Furthermore, the performance of the models is measured using 21 features, eight features, and seven features. For performance testing, it uses the SVM method. The weighting process is carried out in the cross-validation table to overcome unbalanced data sets. The first testing uses seven selected features from the Chi-Squared, GR and IG methods. While eight features testing adds the X6 variable (Gross Profit Margin). Table 4 and table 5 use splitting datasets, 60%, and 40 %. The tested data consisted of 48 company data, 40 non-bankrupt companies, 8 bankrupt companies. Table 4 and table 5 use splitting datasets, 60%, and 40 %. The tested data consisted of 48 company data, 40 non-bankrupt companies, 8 bankrupt companies. In table 4 and table 5, the solvent TP represents the percentage of the solvent companies and those detected correctly not bankrupt (True Positive). True Negative (TN) of bankruptcy is a percentage of bankrupt companies that successfully detected.
The use of feature selection in the prediction of 1-year bankruptcy can increase the accuracy produced. However, for predicting the accuracy of bankruptcy accuracy of 2 years before, the highest accuracy value obtains 93.8%, and it happens using all existing features. The linear kernel function is suitable for predicting bankruptcy one year before while for predicting bankruptcy two years prior uses the Radial Basis Function (RBF) as a kernel function. Although the unbalanced data set tried to overcome by weighting the cross-validation table, but this method only succeeded in detecting bankrupt issuers at a maximum of 75%. Table 4 shows that the best result is achived using the SMO method, 7 features. The best method to predicted 2 years before bankruptcy is the SMO method, 21 features. This result is consistent with Alaka's research [19]; which states that the SMO as a method suitable for use in case of data sets with small amounts. Additionally, SMO can be implemented along with the existing filtering feature selection method. Although Lin [20] recommends using the RBF function on SMO, this result only applies to predictions of bankruptcy 2-years before, while predicting 1-year bankruptcy it simply uses linear kernel functions. The possibility of this happens because for the prediction of 1-year bankruptcy has a shorter time so that the factors that determine bankruptcy are fewer. Furthermore, for the prediction of one year of bankruptcy, the characteristics of the company predicted to have a tendency can separated linearly. As can see at Table 4

Conclusion
This research has shown that the best method used in bankruptcy prediction cases for company data from the Indonesian Stock Exchange is Sequential Minimal Optimization (SMO) method. The issuer's data used needs to normalize first. The optimal prediction of 1-year before uses feature selection and linear kernel functions. The best way to predict 2-years before is not using feature selection and using the RBF function. There are 7 ratios influence the determination effect of 1-year before bankruptcy. Those are PER, Operating Profit, Net Profit Margin, Basic Earning Power, ROE, ROA, Equity to Assets. For the problem of unbalanced data sets, it still needs to be reviewed using different techniques; one of them is by using methods to detect outliers by assuming that the issuer is bankrupt as an outlier.