{'fraction_non_alphanumeric': 0.02350851530665552, 'fraction_numerical': 0.03667328387838262, 'mean_word_length': 4.5813411078717206, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Data-Driven Modeling of Ultimate Load Capacity of Closed- and Open-Ended Piles using Machine Learning

7 Field pile load tests are fairly expensive experiments that can be applied to certain pile types required to 8 be installed in full scale. Hence, it is neither practical nor efficient to perform a load test for every 9 installed pile. While there exist many empirical relations for predicting pile capacities, such methods 10 typically suffer from accuracy and generality. Therefore, current geotechnical practice still looks for 11 methods to accommodate full-scale pile load testing to serve as both accurate and practical tools. In this 12 study, load bearing capacities of closed and open-ended piles in cohesive and cohesionless soils are 13 predicted using machine learning. Nine such methods are utilized in the analyses where CPT and pile 14 data are considered as the learning features necessary to teach those methods the database gathered via 15 a comprehensive search. Then, machine learning models are developed, and the databases are separated 16 into five-folds according to the cross-validation-principle, which are used for both training and testing 17 of the machine learning methods. Model predictions are validated with classical empirical equations. 18 Results indicate that the Relevance Vector Regression and the Random Forest methods typically 19 generate considerably better predictions than the other methods and empirical equations. Hence, 20 machine learning methods are found as reliable tools to predict the pile load capacities of both open- 21 ended and closed-ended piles provided that there is a large enough database and an appropriate method 22 to use. 23


Introduction 27
With the extent of advanced computer technology, machine learning has become one of the hottest topics 28 in computer science that are increasingly being applied to solving civil engineering problems in the last 29 few decades (Reich 1997  and  show better predictive performance than other methods used by (Abu-Farsakh and 41 Titi 2004). Also, Schmertmann-Nottingham method (Schmertmann 1978) provides reliable results on 42 the load capacity of piles. Another study on the subject is Eslami and Fellenius (1997) which proposes 43 a direct method named UniCone to estimate pile bearing capacity from CPT with pore pressure 44 measurements (CPTU). Thereafter, an enhanced procedure is proposed to improve this method (Niazi

Machine Learning Methods 102
Machine learning (ML) is an efficient data-driven approach used to model the relationship between input 103 and output of a system. In order to develop such a representative model, a bulk of data obtained from 104 experiments or observations is represented with a high dimensional feature vector, which is then mapped 105 onto an output-space via a regression function. The weights of the parameters in the regression function 106 are estimated by minimizing a loss function defined by the corresponding MLM. Linear regression 107 methods provide an explicit predictive model, making them more interpretable than the nonlinear 108 methods and allowing for analysis of parameters' effects on the model output. On the other hand, 109 nonlinear methods provide an implicit function such as a black-box model achieving higher performance 110 than linear methods, but with less interpretability. This is especially the case when a nonlinear behavior 111 exists in the input-output variables.

112
In this study both linear and nonlinear regression methods are used, which are summarized in the 113 following subsections. We focus more on providing a general understanding of the regression methods 114 rather than a rigorous theory and technical details. Interested readers are referred to related references        Eslami and Fellenius (1997). Geometric mean yields more accurate and 254 repeatable values to analyze the CPT results as opposed to the arithmetic mean since it is less affected 255 by the outliers. The influence zone is defined as the distance measured from 4D below the pile tip to 8D 256 above it when the soils above the pile tip are weaker than they are below. Here D is the pile diameter. If 257 the soil above the pile tip is dense, then the influence zone starts from 2D above the pile tip which 258 decreases the effect of denser soils lying above. Figure 1a and 1b show the interpretation of the (qc)end 259 feature when the soil above the pile tip is weaker and denser, respectively. Other input features, (qc),ave 260 and (fs),ave , are determined by taking the geometric average of measured cone end resistance and sleeve 261 friction resistance along the pile length. The first 2D distance is not considered due to soil disturbance.

262
These processes are shown in Figure 1c   where is the atmospheric pressure. It is also observed that the mechanical cones measure the sleeve 279 friction resistance two or three times larger than electrical cones. Therefore, the sleeve friction readings 280 taken from mechanical cones are converted down to electrical readings by a factor of two. Once all the 281 input are prepared, feature scaling is applied to eliminate the dimensions. All the feature variables are 282 reduced to suitable forms by applying a zero-mean standardization technique as follows: where is the initial feature variable; ̅ is the mean of the feature vector and  is standard deviation. 285

Training and Validation 308
Once the database is obtained and input-output variables are determined, the database need to be trained 309 and validated. In this study, the k-fold cross validation technique is used to both train the database and

317
The training dataset is randomly divided into 'k' number of mutually exclusive subsets (folds) of 318 approximately equal size. The MLM are trained on k-1 folds, and the remaining one is used for testing 319 the performance of the learning model. This process is repeated k times in a way that each fold is used 320 for testing only once. Expected performance is estimated by averaging the measure, which could be 321 either in terms of the predictive error or predictive accuracy of the learning method over the k number 322 of trials. The advantage of this method is that each sample is used only once in the test dataset and used 323 k-1 times in the training dataset. When k is chosen as equal to the number of training samples, this refers 324 to 'leave-one-out' cross validation, which is more computationally expensive than the k-fold cross 325 validation due to the need to run the algorithm a total of k times. Note that a less amount of bias and 326 high variance occur in the predicted results as the number of k increases. Therefore, an appropriate k 327 value needs to be chosen depending on the size of the dataset to provide statistically significant results.  where and ̂ correspond to the actual and the predicted value of the i th sample, respectively, while ̅ 336 refers to the mean of the actual data. R 2 takes values between 0 and 1, where the closer it is to 1 the 337 better the prediction (goodness of fit) gets. It is widely acceptable that there is a strong correlation if the 338 correlation coefficient is greater than 0.8, which means R 2 gets larger than 0.64 (Sadrossadat et al. 2013).

339
RE is the ratio of the difference between the predicted and actual value to the actual value as below:

459
The methods are first applied to the initial database including the fs feature whose results are presented 460 in

508
Following this third analysis, now named 219S_4F_4ML, the results are compared for R 2 and RE 509 distributions in Table 5, and its boxplot given in Figure 5.   Table   540 6 and its boxplot graph is in Figure 6. 541 542 543

557
The second set of data used in 49S_5F_9ML analysis is prepared by eliminating 11 pile test data with 558 long and large diameter piles since these piles have considerably large shaft capacities causing an outlier 559 behavior. The results of the analysis are given in Table 7 and the boxplot graph can be seen in Figure 7.   (Table 5), since the mean values are not increased significantly in the expanded database 575 we focus our attention back on the second set of analysis with 120S_4F_4ML which seems to present a 576 more consistent dataset herein. Thus, Figure 8 presents the actual pile capacities versus the predictions 577 for all folds in this second round of analysis. The best performance of the MLM is observed when folds 578 1, 2, 3, and 5 are used for training, and fold 4 is used for testing. Therefore, the RVR method gives the 579 highest R 2 value of 0.91 and the lowest RE of 0.27 showing that this method presents a better 580 performance than the classical methods. Figure 9 shows predicted versus measured pile capacity graphs 581 in this case. Also, the regression graph of RVR compared to the classical methods is given in Figure 10.   Figure 1 Interpretation of the input features from CPT results; a) (qc)end for the soil above the pile tip is weaker, b) (qc)end for the soil above the pile tip is stronger, c) (qc)ave, d) (fs)ave   Regression graphs of ML methods and traditional methods Figure 10 Regression graph of RVR method compared with traditional methods, (R 2 = 0.91 for RVR, 0.72 for Schmertmann, 0.74 for DeRuiter, and 0.84 for LCPC) Figure 11 Regression graphs of 49S_5F_9ML, (a) MARS, (b) RF, (c) GPR, (d) VHGPR, (All folds are used as testing data) Figure 12 Regression graph of VHGPR method compared with traditional methods, (R 2 = 0.96 for RVR, 0.83 for UWA, 0.78 for NGI, and 0.69 for Fugro)