{'fraction_non_alphanumeric': 0.03249326951839665, 'fraction_numerical': 0.01473227639844451, 'mean_word_length': 4.889671878440872, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Modeling of Electrical Resistivity of Soil Based on Geotechnical Properties

Determining the relationship between the electrical resistivity of soil and its geotechnical properties is an important engineering problem. This study aims to develop methodology for finding the best model that can be used to predict the electrical resistivity of soil, based on knowing its geotechnical properties. The research develops several linear models, three non-linear models, and three artificial neural network models (ANN). These models are applied to the experimental data set comprises 864 observations and five variables. The results show that there are significant exponential negative relationships between the electrical resistivity of soil and its geotechnical properties. The most accurate prediction values are obtained using the ANN model. The cross-validation analysis confirms the high precision of the selected predictive model. This research is the first rigorous systematic analysis and comparison of difference methodologies in ground electrical resistivity studies. It provides practical guidelines and examples of design, development and testing non-linear relationships in engineering intelligent systems and applications.

• The models demonstrate significant exponential negative relationships.
• Artificial neural networks show much greater accuracy than other models.

Introduction
Investigations of the electrical resistivity of soil are important for various industrial applications, for example, estimation of corrosion of underground facilities, analysis of power distribution grounding systems, safe grounding design, see, for example, (Ackerman et al., 2013;Zhou Wang et al., 2015;Zastrow, 1979;Datsios et al., 2017;Clavel et al., 2018;Gomes et al., 2018), and the references therein.
The electrical resistivity of geo-materials has attracted the attention of the researchers since 1912 when Schlumberger used the electrical resistivity measurements to characterize the subsurface ground (Griffiths & King, 1965). Since then, the need to understand the behavior of electrical resistivity of soils has been extended to different areas such as design of electrical earthing system, monitoring moisture content for agriculture applications, electro-osmosis consolidation and drainage of soils, and assessing the homogeneity of compacted clay liner (Laver & Griffiths, 2001;Lim et al., 2013;Rhoades et al., 1976;Abu-Hassanein et al., 1996;Tabbagh et al., 2002;Brillante et al., 2015;Jones et al., 2011;Al Rashid et al., 2018).
In most of these areas the number of measurements can be substantial. Even for a few geotechnical soil properties their composition can often result in cases that are not reported in previous studies or available technical documentation. It requires an automatic computer system that can deal with these issues to determine a level of electrical resistivity to be used in practical decision-making solutions. This article discusses and compares several design approaches in this inference analysis.
For engineering design purposes, as the electrical conductivity of soils is a function of its geotechnical properties (soil mineralogy, particle size distribution, void ratio, pore size distribution, pore connectivity, degree of water saturation, pore water salinity, and temperature), it is important to determine a robust quantitative relationship between the electrical resistivity of soils and its geotechnical properties, see (Kibria & Hossain, 2012).
In the vast majority of applications determining soil electrical resistivity is done by trial and error. The results are often heavily based on empirical knowledge and can be influenced by subjective perceptions. There is a need to develop rigorous automatic methodologies allowing to estimate electrical resistivity with more certainty. To achieve this, it is important to evaluate various alternatives approaches.
Several laboratory studies have collected data on electrical resistivity of soils and their physical properties, see, for example, (Edlebeck & Beske, 2014;Zhou Wang et al., 2015;Southey et al., 2015;Al Rashid et al., 2018;Alipio & Visacro, 2013Mokhtari & Gharehpetian, 2018). However, only a few electrical resistivity models were used to assess the role of different physical properties.
Unfortunately, none of these articles justified their model selection. No rigorous statistical evidence was found that can help to determine the best model. The aim of this study is to develop several statistical models of electrical resistivity and to provide general methodology for their comparison and performance evaluation. This systematic approach helps in selecting the most powerful predictive model.
It is worth mentioning that while artificial neural networks and other advanced computational methods were used for modeling characteristics of materials, see, for example, (Gençoglu, M.T. & Cebeci, 2009;Gusel & Brezocnik, 2011;Hwang et al., 2010), this research is the first rigorous systematic analysis and comparison of difference methodologies in ground electrical resistivity studies. This paper provides practical guidelines and examples of modelling and investigating non-linear relationships in engineering applications.
Extensive experimental study was conducted in the Department of Engineering at La Trobe University to investigate the effect of soil minerology, moisture content, water salinity, and dry unit weight on the electrical resistivity measurements. These experimental data were used to develop a linear regression model, an exponential non-linear regression model, multivariate adaptive regression splines (MARS), and the artificial neural network model (ANN) for predicting electrical resistivity of soil. Several other models were investigated, but are not reported in this paper as their performances were not satisfactory.
While this paper concentrates on the design, development and testing of non-linear models in engineering soil studies the presented methodology can be applied to much wider classes of intelligent systems. In particular, the discussed ANN models can provide the overall trends in rather complex relationships even from the limited data.
All computations, data transformations, visualisations, statistical analysis, and models fitting were performed using R 3.5.0 software, in particular, the packages plyr, nlme, car, hexbin, MASS, earth, nnet, and devtools. The data and R codes used in this article are freely available in the folder "Research materials" from https://sites.google.com/site/olenkoandriy/.

Data
The electrical resistivity measurements were obtained using a soil box shown in Figure 1. The measurements process was conducted using the Wenner fourelectrode method, (Wenner, 1915;ASTM G57, 2006). A DC test voltage with the magnitude 12V is applied between the outer electrodes, where the corresponding current, I, and the voltage drop between the inner electrodes, V, are measured. The resistivity, ER, is then ER = 2πaV /I. See more details about the measurement process in (Al Rashid et al., 2018).
The full data set consists of 864 observations with five variables. The variables of interest for this study are: • Electrical resistivity (ER): This is a continuous variable that measures the electrical resistivity of soil in Ohm-m (the response variable). • Soil moisture content (Moist): This is a continuous variable that measures the percentage of gravimetric water content. It is the ratio of weight of water to the weight of solid soil particles.
• Dry unit weight (Uw): This is a continuous variable that measures the dry unit weight in kN/m 3 .
• Soil type (ST): This is a factor (categorical) variable that identifies the soil type. Nine different soil types were test in the conducted experimental study. They comprise different mixtures of Kaolin-Bentonite (K-B) and Kaolin-Sand (K-S) as shown in Table 3 which also provides the number of observations for each type of soil. The engineering properties of Kaolin, Bentonite, and Sand used in this study can be found in (Al Rashid et al., 2018). Table 1 provides a snapshot of a part of the original data frame for only two soil types. Namely, pure kaolin (100%K) and kaolin with 10% of bentonite (90%K+10%B). Table 2 shows the summary statistics of all continuous variables, namely electrical resistivity, molarity, moisture and unit weight. Table 3 provides the number of observations for each type of soil. A histogram of the electrical resistivity of soil is shown in Figure 2a. This distribution is highly skewed to the right, indicating that ER is abnormally distributed (potentially exponentially distributed). However, the histogram of  log(ER), which is shown in Figure 2b, is much closer to the normal distribution. Therefore, the log or similar transformations of the ER variable might be appropriate when fitting the linear regression model.

Ordinary least squares approach (OLS)
First, the function lm of the R software was used to fit the multiple linear regression model to the data set, where the electrical resistivity of soil is the outcome variable, and the predictors are molarity, moisture and unit weight  in addition to the soil type (where this variable is treated on the base of the percentage of kaolin that is mixed with bentonite and sand). Figure 3 displays that the electrical resistivity for 100%K composition is likely to yield variation higher than those from other types of soil. There is a big difference in variability of ER for the soil that consists of pure kaolin and the variability of ER for soil that mixed with either bentonite or sand. This may indicate that treating the soil composition variable on the basis of pure and not pure kaolin leads to an improvement in our liner model. Hence, we fit the following three models: It can be seen that the adjusted R 2 for model (1) (2), these two variables are replaced by ST.K which results in an increase in the adjusted R 2 from 16.23% to 20.57%. This means it is better to treat the soil composition variable on the basis of whether it is mixed with bentonite or sand, or not. Moreover, the adjusted R 2 increased from 20.57% to 42.12% (Model (3)). Hence, the interaction effect between soil type and Moisture (ST.K × M oist) and interaction effect between soil type and unit weight (ST.K × U w) variables are significant predictors. Model (3) explains 42.12% of the variability in the electrical resistivity of soil using this data. However, the validity of inference from this model depends on the linear regression assumptions being fulfilled. A violation of the underlying assumptions may lead to an invalid conclusion and an opposite conclusion can be obtained when using different samples (Montgomery et al., 2012). The main assumption is that the errors are from a normal distribution (ε ∼ N (0, σ 2 )). So, we check error normality and constant error variance assumptions. Figure 4 illustrates the residuals vs fitted plot and normal Q-Q plot. A normal Q-Q plot raises concerns over the underlying normality. In addition, the residuals vs fitted plot shows that the constant error variance assumption is violated. There seems to be an increase in error variance as the fitted values increase. Hence, the detection of such problems undermines the validity of first simple linear models.

Box-Cox transformation
The relationship between the dependent and independent variables is not a linear relationship. This may be the reason behind the problems that were found in the previous models in addition to the small value of R 2 .
The non-normality in the response variable often causes issues in the OLS technique. A violation of this assumption may lead to a wrong conclusion. Therefore, this problem needs to be addressed before using the OLS technique. One way to deal with non-normality in the response variable is using Box-Cox transformation. In the data set, all values of the response variable are positive (ER > 0). According to (Sakia, 1992), for positive values of the response variable, (y i ), the family of the Box-Cox transformation is defined as: The parameter λ can be estimated using the profile likelihood function. The dependent variable was transformed using the Box-Cox transformation. The estimated value of λ was very close to 0 (λ = −0.05). Consequently, the log of the ER variable might be an appropriate transformation. So, the natural logarithm was applied to the dependent variable and the models were refitted again as it is shown below.
The R-squared and adjusted R-squared values significantly increased for these three models which is a substantial improvement of the models.
To check the validity of the assumptions the residual vs fitted plot was examined. For model (6), the results are shown in Figure 5. The residual vs fitted plot shows that there is no dependency between residuals and fitted values which indicates that there are no violations to errors normality. Additionally, the normal Q-Q plot for model (6) shows that the points fall close to the 45degree reference lines with a small deviation at the upper tails. Therefore, the normality is probably a reasonable assumption.

Nonlinear regression model
Fitting a linear model provides a reasonable first approximation. However, the relationship between the electrical resistivity of soil and its geotechnical properties is not a linear relationship. Therefore, the use of a non-linear regression model may result in a better fit of the data. Furthermore, using a non-linear model makes the interpretation of the estimated parameters clearer.
According to (Fox & Weisberg, 2010), the general form of non-linear regression models can be written as The best estimate of θ minimizes the residual sum of squares. Unlike linear models, one needs to use an iterative numerical procedure to estimate θ. The non-linear least-squares algorithm may not converge if inappropriate starting values are provided. Therefore, appropriate starting values need to be selected (Fox & Weisberg, 2010) to estimate the parameters. Due to the nature of the data (see the discussions in the previous sections), the non-linear model that may fit the data well is This model was fitted using the nls function of the R computer package nls. The appropriate starting values can be obtained by fitting a linear model to the data and taking its coefficients as starting estimates. The starting values to estimate β 0 , β 1 , β 2 , β 3 , β 4 , β 5 , β 6 and β 7 in model (7) were obtained from The estimates of the coefficients, together with their corresponding standard errors, observed test statistics and p-values of Model (7) suggest that the predicted value of ER is given by Unfortunately, the predicted ER using this model cannot be less than the value of the intercept (44.63). However, we have some actual values of ER that are smaller than this value. As a result, this model will fail to accurately predict electrical resistivity when its actual value is less than 44.63. In addition, there is a positive relationship between ER and Mol (since the estimate of β 4 > 0) which contradicts to the previous models. Therefore, we refine the functional relationship between the response and the predictors and obtain the estimated equation We also refit the model using the ST.K variable instead of ST.KB and ST.KS. Namely, we fit the following model The R output below demonstrates that all variables in model (10) Figure 6 shows the predicted vs actual plots for each non-linear model. Plots a, b and c are for models (8), (9) and (11)  The MSE values of models (8), (9) and (11) are 41471.40, 40908.61 and 36089.50 respectively. Hence, the best of these models is model (11) since it has the smallest value of AIC and the smallest value of MSE. Figure 7 shows the residuals vs fitted values plot for model (11). Overall, there is concern about the dependency of errors and the magnitude of residuals increases with the increase of fitted values.

Multivariate adaptive regression splines
The article (Friedman, 1991) introduced multivariate adaptive regression splines (MARS) which is a flexible regression model that can be used to handle both linear and non-linear relationships. The significant feature of MARS lies in its ability to generate simple and easy-to-interpret models that capture mapping in multi-dimensional data (Zhang & Goh, 2016).
The MARS model has the form (Hastie et al., 2009): where h m (X) represents a basis function (hinge function) or product of two or more such functions. Each basis function consists of the pair max(0, x − c) and max(0, c − x) where c is a knot. Each pair is multiplied by a parameter which is estimated by minimizing the residual sum of squares (RSS).
The MARS model divides the data into regions and fits each region with an appropriate model. Now, we refit models (4), (5) and (6) using the MARS approach. The MARS models were fitted by using the R package "earth". The results of the fits to the ER data are available in the folder "Research materials" from https://sites.google.com/site/olenkoandriy/. Figure 8 shows the predicted ER vs the observed ER for the three MARS models. To compare these MARS models and to check their accuracy, the MSE for each model was calculated. Table 4 shows the obtained MSE values.   Figure 9 shows the residual plots of the second MARS model. It can be seen that the residuals vs fitted plot shows no dependency in errors and they appear to have almost constant variance. Furthermore, the residuals QQ plot shows that the errors are normally distributed.

Artificial neural network model
The artificial neural network (ANN) can learn from data and detect patterns and relationships in data (Agatonovic-Kustrin et al., 2000). ANNs can be used to predict the output from available information. The neural structure of an ANN consists of a number of neurons which are known as processing elements (Erzin et al., 2010), where each processing element represents an equation that consists of weighted inputs, transfer function and one output (Agatonovic-Kustrin et al., 2000).
We explore the use of ANNs to predict the electrical resistivity of soil based on its geotechnical properties. Three ANN models (ANN.model1, ANN.model2 and ANN.model3) were developed for predicting ER.
The first model, ANN.model1, takes into consideration the effect of ST.KB, ST.KS, Mol, Moist and Uw, whereas the second model, ANN.model2, takes into account the effect of ST.K, Mol, Moist and Uw. The third model, ANN.model3, takes into account the effects that are in ANN.model2 plus the effect of interaction between ST.K and Moist, and the interaction between ST.K and Uw. All three models have the output ER.
In most applications, a single hidden layer is enough for reasonable approximation (Agatonovic-Kustrin et al., 2000). So, one hidden layer is chosen for each model. (Erzin et al., 2010) pointed out that the upper limit for the number of neurons in the hidden layer is equal to 2i + 1 where i is the number of input parameters. Hence, the number of neurons in the hidden layer for ANN.model1, ANN.model2 and ANN.model3 should not be greater than 13, 11 and 15, respectively. To get the optimum number of neurons in the hidden layer of each model, we fitted each model with one neuron in the hidden layer and then increased the number of neurons to the upper limit. Moreover, we randomly divided the data into a training dataset (75% of the original data) and an independent validation dataset (25% of the original data) to estimate the performance of the models.
The neural network plots for each model are shown in Figure 10, 11 and 12. Note that each model has two bias layers (B1 and B2).    Figure 13 shows the predicted value of ER vs actual values for each model. It can be seen that most of points lie very close to the reference lines, especially for ANN.model2. The symbols represent the training sample and the • symbols represent the validation sample. The MSE errors for ANN.model1, ANN.model2 and ANN.model3 are 182060.6, 49255.5 and 63702.84 respectively.
We can conclude that ANN.model2 is the best model and it can be used to reliably predict the electrical resistivity of soil.

Comparison and validation of models
We explored various statistical models to fit the electrical resistivity of soil and its geotechnical properties. The methods that were used include linear, non-linear regressions and artificial neural networks (ANN). Each model has its own advantages and disadvantages. Unfortunately, there is not single statistic that can be used to choose which one of these models is the best. Therefore, a particular statistic was used for each method. For example, the R-squared was used to compare linear models fitted using OLS approaches, and MSE and AIC were used to compare non-linear models.
To confirm our findings and compare all models we used the cross-validation analysis. The data were randomly divided into a training dataset (75% of the original data) and a validation dataset (25% of the original data). Then, the best four models, and the artificial neural network model ANN.model2, which input layer consists of ST.K, Mol, Moist and Uw, were refitted using only the training dataset. The new fitted models were applied to the validation dataset to predict ER. MSE was calculated for each model. The procedure was repeated 100 times. Each time, new training and validation data sets were randomly selected. The mean of MSEs for each model was calculated, see Table 5. The result shows that the most accurate prediction values of the electrical resistivity were obtained using the ANN model. Based on all the all above evidence we conclude that the neural network model (ANN.model2) is the best model for predicting electrical resistivity.

Conclusion and future work
The main objective of this study was to develop an accurate statistical model that can be used to estimate the electrical resistivity of soil based on the knowledge of soil type (ST), molarity (Mol), moisture (Moist) and unit weight (Uw).
The obtained results showed that there is a significant exponential negative relationship between electrical resistivity and all four predictors. Moreover, it was demonstrated that treating soil composition as a factor variable (denoted by ST.K) on the basis of pure kaolin or not pure kaolin is better than treating it as a continuous variable based on the percentage of bentonite (ST.KB) and sand (ST.KS). The interactions between ST.K and moisture, and between ST.K and Uw were found to be significant predictors for first two models. As a result, both these interactions were included in the final models. Moreover, the log transformation of the dependent variable yields better models. In particular, it substantially improved the value of R-squared in the linear models from about 0.20 to more than 0.80.
The analysis was performed to identify the optimal combination of the degree of interactions and the number of retained terms or complexity of neural networks. For MARS2 and ANN2 models it was shown that the interaction between ST.K and Uw does not contribute significantly to electrical resistivity prediction. A plausible explanation is that MARS2 and ANN2 models use more flexible non-linear transformations compare to the second model. They can reduce prediction errors directly via separately transformed ST.K and Uw.
Several useful models have been developed and the results from each model were compared with the actual values of electrical resistivity using the crossvalidation. In addition, performance indices such as the coefficient of determination (R-squared), MSE and AIC were used to assess the performance of the models.
It was found that the ANN model is able to efficiently predict the electrical resistivity of soil and is better than the other models that were developed. The developed ANN model in this study could contribute to the continuous research effort in the field of electrical resistivity imaging as it captures the influence of different geotechnical properties of soil on its electrical resistivity. Furthermore, it sheds light on what has been learnt throughout the study in terms of the sensitivity of different soil variables and how its incremental changes in the experimental program should be chosen to maximize the accuracy of the developed model. In this regard, it is recommended that, in the future experimental studies, the incremental change in the percentage of bentonite and sand, salinity of pore water, moisture content, and dry unit weight should be smaller than the values used in this study. In fact, this recommendation can be justified by the observed exponential relationships between electrical resistivity of soil and its geotechnical properties which causes a large change in electrical resistivity even with a small change in geotechnical properties. However, experts in the field know that very small incremental changes in the geotechnical parameters of soil in laboratory experiments are often difficult to be achieved accurately. Therefore, their effects on measurement results could be lower than the true ones. It is desirable to implement experimental designs that balance the size of increments and the number of repeated measurements. The problem of balancing and assessing accuracy of such tests is an interesting direction for future research.
For fully automated intelligent system analysis based on the first two models, transformation of data is an area of potential improvement. The choice of transformation method is conventionally based on descriptive statistics results. This process can be semi-supervised if a finite number of admissible functional transformations and relationships is specified.
One of the main known ANN limitations is knowledge extraction from trained ANNs, i.e. interpreting ANN models similar to the other methods that provide functional relation to input effects. The Lek's profile and local interpretable model-agnostic explanations methods, see (Zhang et al., 2018), could be potentially useful approaches in interpreting ANN results.
For small randomly selected test or training sets ANN resulting models can substantially vary, nevertheless, providing rather similar predictions. Studies with a large number of explanatory parameters can also result in a large set of candidate models. In such studies it would be interesting to quantitatively investigate agreement levels between top selected models by using various agreement coefficients, see (Olenko & Tsyganok , 2016).
Finally, in the future research it is important to investigate other parameters like the proctor density of the soil, ambient temperature, two-layer or a multilayer stratification to obtain finer electro resistivity models.