{'fraction_non_alphanumeric': 0.032411288069293095, 'fraction_numerical': 0.00824252584520816, 'mean_word_length': 5.203206239168111, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Prediction of Defect Prone Software Modules using MLP based Ensemble Techniques

Prediction of defect prone software modules is now considered as an important activity of software quality assurance. This approach uses the software metrics to predict whether the developed module is defective or not. This research presents MLP based ensemble classification framework to predict the defect prone software modules. The framework predicts the defective modules by using three dimensions: 1) Tuned MLP, 2) Tuned MLP with Bagging 3) Tuned MLP with Boosting. In first dimension only the MLP is used for the classification after optimization. In second dimension, the optimized MLP is integrated with bagging technique. In third dimension, the optimized MLP is integrated with boosting technique. Four publically available cleaned NASA MDP datasets are used for the implementation of proposed framework and the performance is evaluated by using F-measure, Accuracy, Roc Area and MCC. The performance of the proposed framework is compared with ten widely used supervised classification techniques by using Scott-Knott ESD test and the results reflects the high performance of the proposed framework.


I. INTRODUCTION
Testing is one of the important activities of Software Development Life Cycle which ensures the delivery of high quality product. However, it has been reported that software testing is also considered as one of the expensive activities of development lifecycle as it consumes more resources as compare to any other activities [8,9,10]. To produce high quality product, all the modules of developing software should be thoroughly tested [7,10] and that is why testing require more resources. Testing cost can be reduced if the defect prone software modules are predicted at early stage of development (before testing) [11,12,13]. With this approach the quality of end product would not be affected as only those modules will be thoroughly tested which would be predicted as defective. This prediction belongs to the class of binary classification problem. Many researchers have solved the binary classification problems by using machine learning techniques such as sentiment analysis/classification [14,15,16,17,18,19], rainfall prediction [20,21], detection of DoS attacks [22,23] and software defect prediction [1,2,3], [25,26,27]. Machine learning techniques are categorized into three types: 1) supervised, 2) unsupervised, and 3) hybrid. Supervised machine learning techniques use the prelabeled/classified data (training data) for training, in which classification rules are developed which are later used to classify the unseen data (test data). In unsupervised machine learning techniques classes are not known in advance so the special algorithms are used to identify the structure/group of data. Finally the hybrid is the integration of both (supervised and unsupervised). Various frameworks and techniques have been proposed by the researchers to predict the defect prone software modules however most of the researchers have used different datasets. This research proposes MLP based ensemble classification framework to predict the defect prone software modules. The framework uses three dimensions to predict the defective modules. In fist dimension, Multi-Layer Perceptron, a well-known Artificial Neural Network is optimized by tuning its parameters. In second dimension, the tuned MLP (tuned in first dimension) is ensemble with bagging technique. In third dimension, the tuned MLP (tuned in first dimension) is ensemble with boosting technique. To validate the proposed framework, four widely used NASA MDP cleaned datasets are used and the performance is evaluated by using: F-measure, Accuracy, Roc Area and MCC. The results of the proposed framework including all three dimensions are compared with the results of a published research [1], in which ten well known supervised base classifiers were used for the prediction of defect prone software modules by using the same datasets which are used in this research (NASA MDP cleaned). The classification techniques with which the proposed framework is compared are: "Naïve Bayes

II. RELATED WORK
Many researchers have used machine learning techniques to predict the software defects at early stage of SDLC. Some of the related studies are discussed here. In [1], the researchers predicted the defect prone software modules by using various supervised classification techniques including "Naïve Bayes (NB), Multi-Layer Perceptron (MLP). Radial Basis Function (RBF), Support Vector Machine (SVM), K Nearest Neighbor (KNN), kStar (K*), One Rule (OneR), PART, Decision Tree (DT), and Random Forest (RF)". The researchers performed the experiments by using twelve widely used NASA MDP datasets and evaluated the performance by using: Precision, Recall, F-Measure, Accuracy, MCC, and ROC Area.
According to researchers the provided results in this study can be used as a baseline for other researchers so that any claim of improvement in prediction with new proposed technique can be verified so, we have compared the performance of our proposed framework with [1]. Researchers in [2] presented a classification framework to predict defect prone software modules. The framework is implemented with two dimensions, 1) with the feature selection, 2) without feature selection. Experiments are performed on cleaned NASA MDP datasets and performance analysis is performed by using: "Precision, Recall, F-measure, Accuracy, MCC and ROC". Performance of the proposed framework including both dimensions is compared with other supervised classification techniques from research [1]. According to results, the proposed framework outperformed other techniques in some datasets. Researchers in [3] analyzed the performance of three well known resampling techniques while predicting the defect prone software modules using same classification techniques which were used in [1]. The used resampling techniques are: "Random Under Sampling", "Random Over Sampling" and "Synthetic Minority Oversampling Technique (SMOTE)". NASA cleaned MDP datasets were used for experiments and the performance is analyzed by using: Fmeasure, Accuracy, MCC and ROC. The performance of all three resampling techniques are compared with the supervised classification techniques used in [1] where no technique was used to remove the class imbalance issue. The results reflected that "Random Over Sampling" performed well among other techniques. Researchers in [4] presented a feature selection based ensemble classification framework to predict defect prone software modules. The framework is implemented on six publically available Cleaned NASA MDP datasets and the performance is analyzed by using F-measure, Accuracy, MCC and ROC. The performance is compared with 10 supervised classification techniques from [1] and the results showed that the proposed framework outperformed all of other techniques. Researchers in [5], presented an integrated technique by using Hybrid Genetic algorithm and Deep Neural Network. Optimum feature set is selected by Hybrid Genetic algorithm whereas Deep Neural Network is used to predict the software defects. The proposed technique is implemented on various datasets obtained from PROMISE repository and results showed the higher performance of proposed technique when compared to other methods.

III. MATERIALS AND METHODS
This research proposes MLP based ensemble classification framework for software defect prediction which consists of three stages: 1) Dataset selection, 2) Classification and 3) Reflection of results. The proposed framework provides the results with three different dimensions: 1) Optimized MLP, 2) Bagging-Optimized MLP, and 3) Boosting-Optimized MLP.  Table I). Each of the used dataset consists of various features/attributes including the target class. The target class which is also known as dependent attribute is the one which is going to be predicted. This attribute consists of either of the following two values: 'y' and 'N'. 'Y' reflects the defect(s) in the modules and 'N' reflects that the particular module is clean. Other attributes in the datasets are known as independent attributes and are used to predict the output class.
The NASA software defect datasets have been widely used by the researchers to validate the defect prediction models as these are publicly available as researchers of this field can verify the claimed accuracy and can work further for improvements. Two cleaned version were provided by [28] of NASA MDP datasets, DS' and DS''. We have used D'' version, currently available at [29]. Many researches have used these datasets [ The experiments are performed in WEKA, which is considered as one of the widely used data mining tool [24].
Second stage classifies the datasets into defective and non-defective modules by using three dimensions: 1) Optimized MLP, 2) Bagging-Optimized MLP, and 3) Boosting-Optimized MLP. We have used the term 'optimized' with MLP because four of its parameters are tuned in WEKA ( Table 2) to achieve high performance by following the hit and trial method. The input datasets are given to classification models with the ratio of 70:30 (70 % training data and 30% test data). MLP (Multi-Layer Perceptron) is a type of Artificial Neural Network (ANN) which uses back-propagation technique for learning. An ANN is inspired by biological neural system of human brain. These networks are considered as non-linear supervised classifiers which can model complex relations between inputs and outputs. The structure of MLP consists of three portions 1) input layer, 2) hidden layer(s), 3) output layer (Fig 2). At least one hidden layer should be existed in MLP network (number of hidden layers can also be increased).  Ensemble learning technique integrates the multiple classification models (weak learners) under the umbrella of meta-algorithm to solve the one/same problem with higher accuracy. Three meta algorithms are widely used including: bagging, boosting and stacking. In this research we have used bagging and boosting. Bagging focuses on homogeneous weak learners where the classifiers learns in parallel and independent from each other and finally result is combined by following a deterministic averaging process. Boosting on the other hand focuses on homogeneous weak learners where the classifiers learn in a particular sequential order where a base model depends on the previous one and the result is combined by following a deterministic strategy.
Third and final stage of the proposed framework deals with the reflection of results which is discussed in the next section in detail.

IV. RESULTS AND DISCUSSION
Performance of the proposed framework including all three dimensions is evaluated with four widely used accuracy measures such as: F-measure, Accuracy, MCC and ROC. All of these performance measures are generated from the parameters of confusion matrix.
We have compared the results of all three dimensions of the proposed framework with 10 widely used supervised machine learning classifiers in Table 3,4,5,6. The results of the 10 classifiers including: "Naïve Bayes (NB), Multi-Layer Perceptron (MLP), Radial Basis Function (RBF), Support Vector Machine (SVM), K Nearest Neighbor (KNN), kStar (K*), One Rule (OneR), PART, Decision Tree (DT), and Random Forest (RF)" are taken from a published paper [1]. The published paper [1] has used the same datasets and accuracy measures which we have used in this research. It can be seen that OPT-MLP and Boosting-OPT-MLP performed well in F-measure, Accuracy and MCC however in ROC Area RF performed better.
Results of MW1 dataset is shown in Table 4 which reflects that in F-measure MLP and Boosting-OPT-MLP performed well whereas MLP, OPT-MLP, Bagging-OPT-MLP and Boosting-OPT-MLP performed well in Accuracy. In ROC Area, OPT-MLP performed well and in MCC, MLP and Boosting-OPT-MLP both outperformed others.
Results of PC4 datasets are reflected in Table 5 which shows that Bagging-OPT-MLP performed better than others classifiers in F-measures whereas OPT-MLP and Boosting-OPT-MLP performed better in Accuracy and MCC. On the other hand RF performed better in Roc Area.  To analyze the effectiveness of proposed framework, we have performed Scott-Knott ESD test (v2.0) [33] (Fig  4). The package of this test is downloaded from [6]. In this test, Accuracy score of all the classification techniques including three dimensions of proposed model and 10 base classifiers from published research [1] is used as input. The test provided the output in the form of sub groups/clusters of classification techniques along with their ranks. The graphical representation of the Scott-Knott ESD test result is given in Fig 5. It can be seen that the whole group of classification techniques is divided into subgroups along with their specific color. The subgroups are created on according on the basis of Y axis. The subgroups are also shown along with their ranks in Table  7. It can be observed that all three dimensions of the proposed framework performed better than other classification techniques. Instead of all the developed software modules, software engineers are now focusing to test only those modules which are likely to be defective so that overall testing cost can be reduced. This goal can be achieved by predicting the defect prone software modules before the testing activity. This research has contributed by proposing a classification framework for predicting the defect prone software modules. The framework uses optimized MLP and ensemble techniques including bagging and boosting. NASA's cleaned MDP datasets are used for the implementation of proposed framework and to analyze the performance, F-measure, Accuracy, Roc Area and MCC are used. Performance is compared with 10 widely used base classifiers from a published research, moreover Scott-Knott ESD test is performed to effectively compare the accuracy of classification models. According to results the proposed framework including all three dimensions outperformed all other classification techniques. For future work, it is suggested feature selection and class balancing techniques should be introduced in the framework and experiments should be performed by using more datasets other than NASA's repository.