{'fraction_non_alphanumeric': 0.028011066100187727, 'fraction_numerical': 0.007607943879063333, 'mean_word_length': 4.864136732329085, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Comparison of software defect prediction models based on machine learning

Software defect prediction has been widely used in software system development, among which the method based on machine learning has proved to be more effective. Firstly, the basic framework of the prediction model and the metric elements used in the prediction process are introduced in this paper. Secondly, the three main machine learning-based software defect prediction models (LR, SVM, and BPNN) are analyzed, and finally the prediction effects of the three models are compared and analyzed by using the experimental results of MDP.


Introduction
With the rapid development of computer technology and the increasing demand for intelligent technology products in the society, various types of software companies have continuously upgraded and evolved software products in order to improve their competitiveness in the industry, which makes the scale and complexity of the software system have grown rapidly, and potential risks have also been introduced. While the software brings convenience to people's lives, various failures that occur during operation have also caused many serious social problems. For example, on July 28, 1962, a bug in the Mariner 1 aeronautical software caused the rocket to deviate from its intended orbit when launched, and the mission controller destroyed the entire rocket over the Atlantic Ocean; between 1985 and 1987, a failure of a radiotherapy equipment caused the therac-25 treatment equipment to emit fatal radiation during the treatment process, which directly resulted in the death of five patients, and the rest of the patients were seriously injured; the AT & T network paralysis in 1990, the Intel Pentium floating-point index division in 1993, and the first failure of the Ariane 5 launch vehicle. It can be seen that the potential defects in the software will not only affect the quality of the software, reduce the reliability of the software, but also cause the collapse of the software system, and bring serious economic losses to individuals or enterprises.
The serious consequences caused by software defects have raised more and more software researchers and developers to pay close attention. In order to minimize the impact of software defects, methods such as software testing are applied to the development of software systems. Although this method has been proven to effectively improve the security and reliability of the system, according to Boehm statistics, software testing accounts for 30% to 50% of the development cost of the software system [1]. In the software development cycle, it needs to invest a lot of time and cost, so exhaustive testing of each software module is unrealistic. At the same time, software testing technology will also be limited by test resources. In the 1970s, starting from the repositories, some researchers used statistics and machine learning methods to predict the number and types of defects in the software system based on the software metrics and the discovered defects, and obtained abundant research results. This method is the software defect prediction, which can be roughly divided into static defect prediction methods and dynamic defect prediction methods [2], among them, the static prediction methods are mainly based on the defect related metrics (relevant data information obtained by quantifying software code as static characteristics). Through statistical analysis of these data, a prediction model is constructed to predict the defect tendency, defect density or number of defects of the program module; while the dynamic defect prediction method is to predict the distribution of system defects over time by analyzing the time of defects or failures, and to discover the distribution of software defects with their life cycle or the time relationship of some stages. At present, researchers pay more attention to static prediction. Therefore, static prediction has also become the dominant method in software defect prediction [3][4][5].
Static software defect prediction methods mainly include machine learning, statistical methods, machine learning + statistical methods, and statistical methods + expert opinions [6]. Machine learning is one of the core research directions in the field of artificial intelligence, and has been widely used in the research of software defect prediction. Software defect prediction methods based on machine learning have also been proven to have efficient prediction performance in several practices. This paper mainly studies and discusses the above method.

Software defect prediction 2.1. Basic framework of prediction model
The software defect prediction model based on machine learning needs to use the marked defect historical data and abstract the metrics from these data, and select the optimal features according to a certain learning rule, so as to train the classifier to predict the unknown samples. The general process of software defect prediction model based on machine learning method is shown in Figure 1.
Software source code, defect report, etc. Software source code, defect report, etc.

Preprocessing
Training data Training data The defect prediction process shown in the figure 1 is mainly divided into four parts: software defect data acquisition, metrics extraction, prediction model construction and performance evaluation [3,7,8] .
(1) Use the version control system and defect tracking system to collect software defect data from the software warehouse that contains all the data of the software project development cycle.
(2) The data information obtained in (1) is represented by different software metrics, that is, the data is preprocessed, and then the program modules in the data are marked according to the actual requirements (the attribute measurement of the corresponding module is given defect category information); in order to avoid redundancy and overfitting caused by a large amount of data information, an appropriate number of feature data sets can be selected after labeling to reduce the computation and improve the performance of the classification model.
(3) Build a software defect prediction model in the adjusted data set by using appropriate learning algorithms and statistical techniques; for a new test module, it is necessary to extract its software metrics, then use the prediction model to predict, and finally output the final result of the prediction.
(4) After the prediction results are obtained, the prediction model is evaluated by using the evaluation performance related to confusion matrix. The evaluation results of the model are given through different performance evaluation indexes (accuracy, precision, recall, F-measure, etc.).

Software metrics
The updating of software products has stimulated the development of defect prediction technology, the most typical of which is that a large number of metrics are introduced into the software life cycle. IEEE gave a definition of software metrics in 1997: "Metric is a function, its input is software data, and its output is a single value, which can be used to explain the extent to which a given attribute of software has an impact on software quality. Software metrics are quantitative measurements of attributes that affect software quality " [9] .
At present, Halstead metrics and McCabe metrics are the most basic of all metrics. In the 1990s, with the rapid development of object-oriented development methods, Chidamber and Kemerer [10] proposed CK metrics suitable for object-oriented programs. The following sections focus on these three metrics.
(1) Halstead metrics Halstead studies software complexity from a statistical and psychological perspective, and regards programs as a sequence of symbols composed of executable lines of code. The complexity of the program is measured by calculating the number of operators and operations in the program. The main metrics involved include the length, capacity, difficulty, and workload of the program., among which the basic metrics are as follows:  N1: Overall operator in the program  N2: Overall operands in the program  n1: Different operators in the program  n2: Different operands in the program It is known that H is the predicted length of the program, and N is the actual length of the program. The calculation formula of H and N given by Halstead is: Halstead also gives the calculation formula of program capacity, program level and number of software defects, etc. Let V represent the program capacity, L represents the program level, and B represents the number of software defects. Where, L indicates the ratio of the program capacity in the most compact form of a program to the actual program capacity, reflecting the efficiency of the program. The calculation formula is as follows:  (4) B  N log 2 n1 n2 / 3000  V / 3000 (5) (2) McCabe metrics McCabe proposed the theory of software complexity measurement in 1976. He believes that the complexity of a program can be quantitatively measured by the complexity of the program control flow [11] . The McCabe metrics mainly include multiple metrics such as Cyclomatic complexity, Essential complexity, and Module Design complexity.
Cyclomatic complexity is used to measure the complexity of a module's decision structure. The quantity is expressed as the number of linearly independent paths in the control flow graph G, that is, the minimum number of paths required for reasonable error prevention.
Errors in a program may have a lot to do with high cyclomatic complexity. Its calculation method is very simple, the calculation formula is: V G  e -n  2 . Where, represents the total number of edges in the control flow graph, and n represents the number of nodes in the control flow graph. More simply, Cyclomatic complexity can also be obtained from the number of decision nodes d in the Basic complexity is used to measure the unstructured degree of the program. By simplifying the structured part of the module control flow graph into nodes, the cyclomatic complexity of the simplified control flow graph after calculation is the basic complexity. The calculation formula is: Module Design complexity is used to measure the calling relationship between modules. The higher the complexity, the higher the coupling between modules, and the harder it is to isolate, maintain, and reuse. It is the cyclomatic complexity obtained by removing the decision and loop structure that does not include the calling submodule from the module control flow graph. Module Design complexity is usually much smaller than the cyclomatic complexity.
(3) CK metrics CK metrics mainly describe the degree to which the program conforms to object-oriented design, and includes 6 main metrics [12] :  wmc: Method weight in a class, which represents the sum of the complexity of all methods in a class. Here, CK metrics do not measure the complexity by cyclomatic complexity, but assigns the complexity of each method to 1, So wmc is equivalent to the number of methods in each class.
 dit: Inheritance tree depth, which indicates how many classes a class inherits.  noc: The number of direct subclasses of a class.  cbo: Coupling between objects is a count of classes that are coupled to a class. When a method of a class calls a method of another class, or accesses a variable of this class, we think that the two classes are coupled. The cbo of class A is the size of the collection that refers to A or the class referenced by A  rfc: The size of the response set of the class, that is, the size of all the methods in this class and the collection of all the methods called by the methods in this class.
 lcom: Lack of cohesion in methods. For each method pair in the class, if the pair of methods do not jointly refer to any instance variables, then lcom increases by one, otherwise decreases by one

Software defect prediction model based on machine learning
The realization of software defect prediction is mainly attributed to the use of machine learning algorithm and statistical methods. The current mainstream defect prediction models include Logistic

Logistic Regression model(LR)
Logistic regression algorithm is actually a classification algorithm. Logistic regression is a non-linear regression method obtained by applying a logistic function on the basis of linear regression.
In machine learning, LR is mainly applied to binary classification, that is, to divide all data into two categories 0 and 1 [13] , a given data set D   x 1 , y 1 , x 2 , y 2 , ,  x n , y n  ， where x i  d , y i  i  1, 2, , n . Given the sample x i , the Logic linear regression model is used to predict the value of y i . If it is assumed that the probability of y i being 1 is required, the specific calculation formula is as follows: Given that the hypothesis function of linear regression is: Since the predicted value of the linear regression model is continuous, the range is [0,1]. We can use the Sigmoid function to convert, let z=θ T x : Generally speaking, when g(z)  0.5 , z is classified as true, otherwise when g z  0.5 , z is classified as false. In the LR model, we know that when it is assumed that the function h  x  0.5 , P y  1 is predicted to be a positive class; otherwise, it is predicted to be a negative class.
Assuming that there are m data in the training set, the loss function constructed by the maximum likelihood method in statistics is (the objective function of LR) : Next, we can use the gradient descent method to find the maximum value of the parameter and classify the unknown samples according to the known samples.

Support Vector Machine model(SVM)
SVM is a classifier algorithm, and its core idea is the principle of structural risk minimization [14] . In the case of a small number of training samples, a better generalized learning performance can be obtained by establishing a classifier that does not depend on the dimension of the sample space [15] .
Given a training dataset on a feature space D   x 1 , y 1 , x 2 , y 2 , , x n , y n  , where x i    n , y i Y  1,-1,i  1, 2, , N  , the basic idea of classification learning is to find a dividing hyperplane in the sample space based on the training set D to separate samples of different categories. The equation for dividing the hyperplane can be expressed by the following formula: The purpose of SVM is to select the hyperplane with the largest geometric interval, that is, to obtain the support vector by finding the optimal solution of the following formula: Where, the slack variable is . By constructing a Lagrangian function: For nonlinear partition problems, we can introduce Gaussian radial kernel functions: In this case, the classification decision function can be transformed into the following form: When a new point x is predicted, it is only necessary to calculate its inner product of the point in the training sample to find the value corresponding to y.

BP
Neural Network model(BPNN) The BP algorithm (Back Propagation) was provided by Rumelhart and Hinton of UCSD PDP team in 1986. The learning process of BP neural network includes the dissemination of positive information and error back-propagation. It is one of the most widely used and successful neural network models and one of the common methods of software defect prediction [16] . The topological structure of the entire network model is composed of the input layer, the hidden layer and the output layer. Its structure is shown in Figure 2. BP neural network training is divided into two stages: (1) The dissemination of positive information: input information is transmitted layer by layer and through the hidden layer from the input layer until the output of each cell is obtained; (2) Error back-propagation: The output layer error starts from the output layer, layer by layer, and back propagates. The error of each unit of the hidden layer can be indirectly calculated, and the weight of the previous layer can be corrected with this error.
It is known that the actual output of the k-th neuron in the output layer is y k , and the input is net k , and the output of any neuron j in the hidden layer adjacent to the output layer is j y .

Model comparison 4.1. Performance evaluation standard
According to section 2.1 of this paper, the last step in the software defect prediction process is the performance evaluation of the prediction model. The final prediction results are classified as defective or non-defective. Therefore, the prediction of software defects is a typical binary classification problem, and a basic confusion matrix can be obtained based on the prediction results, as shown in table 1. In the table, TP (True Positive) represents a true example, indicating the number of actual defective and correctly classified instances; TN (True Negative) represents a true negative example, indicating the actual number of defective and correctly classified instances; FP (False Positive ) represents a false positive example, indicating the actual number of non-defective and misclassified instances, FN (False Negative) represents a false negative example, indicating the actual number of defective and misclassified instances.
Common performance evaluation criteria are as follows: Precision (P): Precision, that is, the proportion of samples in the predicted positive class, which reflects the accuracy of the prediction model, also known as the Precision ratio. The calculation formula is as follows: Recall (R): Recall rate, which represents the proportion of correctly classified positive examples to actual positive examples, and reflects the probability that a defective module is correctly predicted. The formula is as follows: In the training process, this paper uses 10-fold cross-validation to calculate the performance results. That is, the training data is divided into 10 parts, 9 parts are used for model training, and the remaining 1 part is used as the test model. In order to reduce the error caused by randomly dividing the data set in cross-validation, each group of experiments was repeated 10 times, and the average value was taken as the final performance prediction result.
For the five data sets in table 2, we mainly use the three machine learning methods (Logistic Regression model, Support Vector Machine model, BP Neural Network model) introduced in Chapter 3 to predict their defects and use the Precision (P), Recall (R) and F-Measure evaluate the performance of the three methods. The final prediction results are shown in table 3. It can be seen from the experimental results that the defect prediction effect of the BP Neural Network model on the five data sets is not ideal compared with that of the other two models. Due to the slower convergence rate in the modeling process, the generalization ability of this method is poor; The Logistic Regression model has better prediction results in the data sets JM1 and MC1 with higher sample size, but that in the data set PC1 with lower data size is significantly reduced. This model is easy to implement in defect prediction, but the prediction results are limited by the sample size; Combining the Precision, Recall and F-Measure value of the five data, the prediction accuracy of the Support Vector Machine model is high, but it is difficult to determine the optimal parameters used in the model, and the prediction ability is unstable.

Summary
This paper mainly introduces the software defect prediction model based on machine learning, and makes a simple analysis of the construction process of the prediction model and software metrics. Logistic Regression model (LR), Support Vector Machine model (SVM) and BP Neural Network model (BPNN) were selected from a large number of prediction models to perform defect prediction on five data sets in MDP data set, and the relevant experimental results are obtained by using Precision, Recall and F-measure. Through the performance evaluation and analysis, the three methods have their own advantages and disadvantages. Therefore, improving the prediction accuracy, reducing the error rate, enhancing the practicality of the model, and overcoming the shortcomings of each model will be the future research directions.