{'fraction_non_alphanumeric': 0.02928830375882465, 'fraction_numerical': 0.007155123068116772, 'mean_word_length': 5.028177113283497, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Prediction of Concrete Properties Using Ensemble Machine Learning Methods

One of the most commonly used materials in civil engineering is concrete; not only is it cheap and strong, but it is also efficient and convenient. The efficiency of concrete is based on the easiness to place and to compact, which is usually known as workability. However, concrete strength and workability works in different ways; hence it is important to divide concrete into two groups: concrete with low workability and concrete with high workability, in order to achieve a more accurate prediction. Since there is a lot of variations of concrete mix designs, the relationship between each mixture is complex and, thus, requires advanced prediction methods in order to find the most accurate relationships between concrete mix proportion and its compression test result.–Recently, many studies have been conducted on applying multiple artificial intelligence (AI) methods in building different complex and challenging prediction models. Thus, this research employs ensemble machine learning methods to precisely forecast compression strength of concrete mix proportion. The accuracy of the proposed method was calculated using two performance measurements. Subsequently, the study has successfully built the prediction model that can accurately map the relationship between concrete mix proportion and compressive strength.


Introduction
In civil engineering, concrete compressive strength is one significant criterion when selecting a type of concrete to be used for a specific purpose [1]. Concrete compressive strength is the ability of concrete to carry loads applied on its surface without cracking, and it is measured with concrete compression test. Concrete will be tested after all the other processes have been completed; making concrete cube or cylinder, curing, and waiting for the concrete to reach its maximum strength (usually 28 days). Any small mistake in the course of testing or making the concrete will result in a repeat of the entire procedure, which could be a slow and expensive process [1].
Since any mistake would require waiting for another 28 days, having the ability to determine concrete strength without having to wait for such time will be a great advantage. Machine learning has been proven to be a better technique than other traditional techniques, due to its incredible learning capabilities [2][3][4][5][6][7]. The models proposed by machine learning are evaluated by two performance measurements, coefficient of correlation (R) and mean absolute error (MAE), to discover the model's accuracy. However, higher water-cement ratio makes the concrete more workable [8], but does result in a lower concrete compression strength. In this research, a classification is done to distinguish between concrete with lower and higher workability, in order to ensure a more precise prediction.
Although machine learning has been proven to be a better technique, it is necessary to choose the best model for every case. This paper, discusses every combination of four models: Linear Regression Analysis (REG), Classification and Regression Tree Analysis (CART), Artificial Neural Network (ANN), and Support Vector Machine (SVM). The purpose of this study is to find the best model for predicting concrete properties, compressive strength, considering its workability.

Linear Regression Analysis (REG)
The REG model that is developed from one or multiple relationship between explanatory variables and dependent variables helps to explain the correlation between the variables and the prediction problems [9]. For instance, changes in the dependent variable Y are always affected by variable X. The formula is as follows: where Y is the dependent variable; X1, X2, …, Xn is the explanatory variables; β0 is a constant variable; β1, β2, …, βn are regression coefficients; and e is the error term.

Classification and Regression Tree Analysis (CART)
The representation of CART model is a tree-like structure. CART model performs classification from selecting input variables, split points, and minimizing the number of branches through repeated operation to minimize total error. The equations are as follows: where i and j are categorical variables in each item; Nj(t) is the recorded number of node t in category j;

Chi-Squared Automatic Interaction Detection (CHAID)
The CHAID model is a decision tree model or statistical model proposed by Kass et al. [10]. In order to generate a decision tree, CHAID uses chi-squared test to determine the optimal and significant splits, where continuous predictors are split into categories with approximately equal number of observations. The model will continue to merge and split until no further splits can be performed, where the group result has no differences. The CHAID model also uses various and different methods to measure different data types. For instances, continuous data from F tests are examined, and categorical data are measured through the CHAID [11].

Artificial Neural Network (ANN)
The ANNs are a group of information-processing models inspired by biological neural networks; ANN model works similarly to that of the human brain, in which neurons are interconnected through synapses [12]. It receives multiple inputs and uses them to make prediction. The processing element has the following characteristics: (1) filtering function to confirm that incomplete data that was inputted to a specific node do not affect the network; and (2) adaptive learning ability to organize the connective weight between nodes. ANNs have multiple input-output systems, and also a basic structure which include an input layer, a hidden layer, and an output layer. The ANNs can be expressed with following equation: where αi refers to ANN activities; ωij is the weight connecting two neurons; oj is an an output signal of the ANN; x is the activation of i th neuron; and σ(x) is the activation function of the ANN that facilitates transformation of inputs into outputs by multiplying the inputs from the processing elements by the corresponding weights.

Support Vector Machine (SVM)
The SVMs are machine learning models that were first proposed by Vapnik in 1995. They are widely used for classification, forecasting, and regression. An SVM model is used when target variable involves categorical data.
where gj(x) is a set of nonlinear transformations from input space; b is the bias term, w is the weight vector estimated by minimizing the regularized risk function.

Performance Measurements
The accuracy of the models is tested using the error indicators, namely performance measurements. This study used two statistical method ti compare the actual and prediction values. The two methods are the coefficient of correlation (R) and mean absolute error (MAE). The performance measurements are explained as follows.
The R index shows the linear correlation between two variables, in this study, the prediction and actual values. The minimum and maximum values of R, are -1 and 1, respectively; are calculated as follows: where Pi is the predicted value; yi is the actual value; and n is the total number of samples.
The MAE is the mean absolute difference between the prediction and actual values; the calculation is as follows: where Pi is the predicted value; yi is the actual value; and n is the total number of samples.

Data preparation
A hundred and three data records of concrete mix proportion from previous research were used to analyze concrete strength. Every data set has seven input variables and two output variables, such as cement, slag, fly ash, water, superplasticizer (SP), coarse aggregate (CA), fine aggregate (FA), slump test result (slump), and 28-day compressive strength test result (fc'), respectively. The data is divided according to their workability level and measured by its slump test result to ensure a more accurate prediction. Concrete mix proportion with lower workability level (slump test result lesser than 12.5 cm) and higher workability level (slump test result equal to or higher than 12.5 cm). The properties and characteristics of the variables of each data set can be seen in Table 1 and Table 2.

The process of training and model selection
The number of data set for concrete mix proportion with slump test result lesser than 12.5 cm and equal to or higher than 12.5 cm are 21 and 82, respectively. Next, each of them is divided into training set (70%) and test set (30%). Training set is analyzed by the numeric predictor, machine learning methods, to determine the pattern which will be used to predict the result from the same input for test set. Later, the result prediction will be compared with the actual result, using two performance measurements: MAE and R. Fifteen different models comprising single and ensemble methods, were used to get the most accurate prediction.

Prediction results and comparison
All of the models were compared to the actual strength, and then ranked based on their performances. The models rank and the result from the best models are mentioned in Table 3, Table 4, and Figure 1.  It is shown on Table 3 that the best model for predicting concrete compressive strength with low workability, slump test result lesser than 12.5 cm, is the Linear Regression (REG) method. However, on Table 4 it is shown that the best model for predicting concrete compressive strength with high workability, slump test result equal to or higher than 12.5 cm, is ensemble Artificial Neural Network (ANN) and Support Vector Machine (SVM) method. Then the prediction of concrete strength is made for every sample in the test set, using the best model for each condition. The comparison between the result from the prediction and the actual value are shown on Figure 1.

Conclusion
It can be concluded from this research, that one of the most important processes of predicting concrete compressive strength is to divide the concrete based on the workability-slump test result. Since workability and strength works in different ways, dividing the concrete will result in higher accuracy predictions. It can be inferred that the best method for predicting concrete with lower workability, slump test lesser than 12.5 cm, is the Linear Regression (REG) method. On the other hand, the best method of prediction for concrete with higher workability, slump test equal to or higher than 12.5 cm, is ensemble Artificial Neural Network (ANN) and Support Vector Machine (SVM) method.