{'fraction_non_alphanumeric': 0.029125154534215693, 'fraction_numerical': 0.013780815940658862, 'mean_word_length': 4.924816889271866, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Neural network approach to reference evapotranspiration modeling from limited climatic data in arid regions

In order to better manage the limited water resources in arid regions, accurate determination of plant water requirements is necessary. For that, the evaluation of reference evapotranspiration (ET0)—a basic component of the hydrological cycle—is essential. In this context, the Penman Monteith equation, known for its accuracy, requires a high number of climatic parameters that are not always fully available from most meteorological stations. Our study examines the effectiveness of the use of artificial neural networks (ANN) for the evaluation of ET0 using incomplete meteorological parameters. These neural networks use daily climatic data (temperature, relative humidity, wind speed and the insolation duration) as inputs, and ET0 values estimated by the Penman-Monteith formula as outputs. The results show that the proper choice of neural network architecture allows not only error minimization but also maximizes the relationship between the dependent variable and the independent variables. In fact, with a network of two hidden layers and eight neurons per layer, we obtained, during the test phase, values of 1, 1 and 0.01 for the determination coefficient, the criterion of Nash and the mean square error, respectively. Comparing results between multiple linear regression and the neural method revealed the good modeling quality and high performance of the latter, due to the possibility of improving performance criteria. In this work, we considered correlations between input variables that improve the accuracy of the model and do not pose problems of multi-collinearity. Furthermore, we succeeded in avoiding overfitting and could generalize the model for other similar areas.


Introduction
Water resources are, for most countries, a key factor in their economic and social development (Sebei et al. 2004). A great challenge for the coming decades will be the task of increasing food production to ensure food security for the steadily growing population. However, the dependency on water for food production has become a critical constraint for increasing food production in many regions that face serious water deficiency (Zhao et al. 2005;Chuanyan et al. 2005;Clemmens and Molden 2007). Hence, according to Naeem and Rai (2005), water shortage requires that new technologies and methods of irrigation be developed that could help in the effective utilization of this precious input. In addition, there is also a need to carry out practices of irrigation water management to achieve high water use efficiency, increase the productivity of existing water resources, and produce more food with less water (Bharat 2006). This necessitates innovative and sustainable research, as well as appropriate transfer of technologies (Pereira et al. 2002).
It should be noted that, in many regions of the world, climate change will increase the average reference evapotranspiration by 2% (De Silva et al. 2007), which will thus increasingly affect cultivation water requirements (Doria et al. 2006).
Further, in Mediterranean regions, irrigation is the only means of producing both high and stable crop yields (Katerji and Rana 2006) and reference evapotranspiration (ET0) is an important quantity for computing the irrigation demands of various crops (Chowdhary and Shrivastava 2010;Dinpashoh 2006). Current irrigation scheduling is based on a well-established crop coefficient and on ET0 procedures to estimate daily crop evapotranspiration (Hunsaker et al. 2007); thus, poor ET calculation appears to be associated with poor estimations of ET0 (Shujiang et al. 2009).
Thus, in order to estimate ET0, much research has been carried out across the world, and a significant number of formulae have been highlighted, but comparison of their results reveals a wide divergence that can reach up to 50% of assessment during the same decade (Smadhi 2000;Lu et al. 2005).
The FAO has recommended the Penman-Monteith method (Allen et al. 1998) because it yields more realistic results (Saidati and Samuel 2006;Hazrat and Lee 2006); yet, this approach has been highly criticized due to its requirement for a high number of meteorological parameters that are usually not available in most meteorological stations.
Given the above facts, we decided to use neural networks to model ET0, based on a daily time step. Using neural networks allows us to use a reduced number of meteorological quantities compared to previous approaches.

State of the art in evapotranspiration modeling
One widely used predictive modeling technique is the multiple regression model (Delacostea et al. 1995). This technique describes the correlation between a dependent variable and a set of explanatory variables, and is based on a statistical analysis in a multidimensional space (Holder, 1985cited in Riad 2003. However, results obtained by this method have often been unsatisfactory (negative value prediction, residue dependence…).
Artificial neural network (ANN) modeling (Eslamian et al. 2008) is a nonlinear statistical technique that can be used to solve problems that are not amenable to conventional, statistical or mathematical methods. Furthermore, the application of up-to-date NN technology allows modeling by black-box NN tools (Aytek et al. 2009).
ANNs are analogous to biological neural networks in that they are highly simplified mathematical models of their biological counterparts. They include the ability to learn and generalize from examples to produce meaningful solutions to problems, even when input data contain errors or are incomplete, to adapt solutions over time to compensate for changing circumstances, and also to process information rapidly (Jain et al. 2008).
The basic unit in the ANN is the node. Nodes are connected to each other by links known as synapses. Usually, NNs are trained so that a particular set of input produces, as closely as possible, a specific set of target outputs (Dechemi et al. 2003). The interest of neurons  (Dreyfus et al. 2004) lies in the properties resulting from their association in networks. The combination of nonlinear functions performed by each neuron and their ability to glean internal information from the data, allows them to perform elementary calculations and recall the knowledge acquired at the learning stage to conduct the classification (Amini 2008).
It should be noted that model result indicate that MLR (multiple linear regression) was also able to predict at a desirable level of accuracy (Deswal and Pal 2008), but NNs are not affected by the multicolinearity problem (Tufféry 2007) that is one of the main problems when developing MLR models (Paulo et al. 2005).
Finally, it should be noted that the general stop criteria for ANN is the problem of overfitting or poor generalization capability. This problem occurs when we have too little data and too precise a model (Sterlin 2007). Several approaches have been suggested in the literature to solve this problem. The simplest is to have three separate databases: a learning database, a test database and a database called "cross validation". Note that this technique requires sufficient data to establish three bases that are at once representative and distinctive.

Materials
Our study was carried out in the region of Adrar, located in the south-west of Algeria. Latitude: 27°49′ N and Longitude: 00°18′ E (Fig. 1). Adrar is characterized by its extreme meteorological parameters.

Climate characteristics
Adrar's climate is dry throughout the year as shown in the ombrothermic diagram in Fig. 2. The climate is characterized by the extended thermal amplitudes during the year, the month and even the day. The absolute maximum temperature reaches 49.5°C in summer (July and August). On the contrary, ice and frosts are rare in this region. Nevertheless, icy days can cause catastrophic damage, especially to traditional farming. Furthermore, the region has recorded: & negligible pluviometry (<25 mm / year). & relative humidity often below 50%; dew is very rare. & a north-east wind blowing almost constantly. & completely clear skies with intense brightness.

Estimation of reference evapotranspiration
The Penman-Montheith equation used for calculating reference evapotranspiration was proposed by Allen et al. (1998): Where ET0 is the reference evapotranspiration (mm day −1 ), R n is the net radiation at the crop surface (MJ m −2 day −1 ), G is the soil heat flux density (MJ m −2 day −1 ), T is the mean of daily air temperature at 2 m height (°C), u 2 is the wind speed at 2 m height (m s −1 ), e s is the saturation vapor pressure (kPa), e a is the actual vapor pressure (kPa), e s − e a is the saturation vapor pressure deficit (kPa), Δ is the slope vapor pressure curve (kPa°C −1 ), γ is the psychometric constant (kPa°C −1 ). The parameters air temperature, sunshine duration, wind speed and relative humidity are taken directly from the meteorological station and are used to estimate other parameters. According to Doorenbos and Pruitt (1977), net radiation and the saturation vapor pressure deficit can be estimated by air temperature and sunshine duration.
Net radiation is the difference between the net short wave radiation (R ns ), and the outgoing net long wave radiation (R nl ). If And Where α is the albedo, R a is the extraterrestrial radiation expressed in equivalent evaporation (mm/day), f(t) is the correction for the effect of temperature on R nl , f(ea) is the correction for the effect of vapor pressure on R nl , and f(n/N) is correction for the effect of the ratio between the actual number and the astronomical number of possible sunshine duration on R nl .
For slope vapor pressure, psychometric constant and soil heat flux, we used mainly air temperature.

Neural network and model evaluation
The neural network is trained with a series of inputs and desired outputs from the training data set. The ANN used in this study is a feed forward network with the backpropagation training algorithm. It is a supervised learning technique used for training ANNs. Basically, it is a gradient descent technique to minimize the squared error between the calculated and desired outputs. The neural network structure in this study had a four-layer learning network consisting of an input layer, two hidden layers and an output layer (Fig. 3). Adjustable weights are used to connect the nodes between adjacent layers and optimized by the training algorithm to obtain the desired results. As noted by Parizeau (2004), the backpropagation algorithm allows training of the multilayer networks. To be useful, this network must have a non linear transfer function on hidden layers and the output layer according to application type-either linear function or non linear function.
This topology uses two sigmoid functions in hidden layers and one linear function in output layer as depicted in Fig. 4.

Description of data and availability
In the present investigation, daily data (temperature, sunshine duration, wind speed and humidity) consist of a series of daily values registered throughout the period of 1,272 days. The registration of these meteorological statements was performed by the meteorological station within the experimental site and was used for estimation of ET0.
Using these observed climatic data, daily values of ET0 were computed initially using the Penman-Monteith (Eq. 1). These computed ET0 values were used to train the ANN models. The database is divided into three subsets: 70% of data are used in the training phase, 15% in the testing phase; the remaining is reserved for validation.
The idea behind this division is to: (1) Take into consideration any seasonal tendency in the ANN model (2) Overcome the overfitting problem.
This also ensures that the statistical properties of the training and testing data are of similar order. As the climatic characteristics of arid zones are important in assessing the applicability of the models in general, the variations in different meteorological parameters in the study area are presented in Fig. 5.
It can be noted that the variability range of meteorological parameters in the study area was very large. For instance, the daily values of temperature ranged between 7.5°C and 41.6°C; relative humidity between 13% and 95%; duration of insolation between 0.00 and 12.30 hours/ day; and wind speed was between 0.00 and 5.09 ms −1 . Hence, any model developed on this data set should have a wide application.

Selection of input variables
A correlation matrix of all input variables is presented in Table 1. This table shows that the linear correlation between temperature and ET0 is 0.86. Hence, any model that uses temperature should be able to estimate the ET0 satisfactorily. The model's accuracy can be improved by considering other variables that have aerodynamic effects on ET0, such as humidity and wind speed. Table 1 also reveals the high correlation between humidity and ET0.
Temperature and humidity are also highly correlated. Therefore, a combination of these two factors may provide a good estimate. Wind speed and insolation are not well correlated with ET0. Nevertheless, these parameters are included in our model for better accuracy of ET0 estimation. It should be noted that all these correlations between variables are of linear type, but the ET0 process is considered to be highly nonlinear.

Criteria of evaluation
The performances of the ANN and MLR models were evaluated to compare their predictive accuracies based on the following statistical criteria: The Nash-Sutcliffe efficiency (E)-proposed by Nash and Sutcliffe (1970)-is calculated by formula (6) according to Krause et al. (2005), and the square value of the correlation coefficient (R 2 ), root mean squared error (RMSE), mean-square error (MSE) and mean absolute relative error (MARE) were calculated as follows: Where E is the Nash-Sutcliffe efficiency Y obs i represents the Penman Monteith observed ET0, Y sim i is the estimated ET0 for the ith values; , Y obs and Y sim represent the average values of the corresponding variable; n represents the number of data considered.
For multiple regression, we added the test of non colinearity parameters using the matrix of covariance, VIF (variance inflation factor), the F statistic and T statistic.
We used the neural network toolbox in Matlab (version 7; http://www.mathworks.com), which has all necessary functions already set, and we programmed all the required equations.

Results and discussion
In order to highlight the necessity of using a neural network, it is necessary to first show the results obtained using MLR.

Multiple regression
It should be noted that the data used in this section are those of the subset of the test phase. The MLR equation used according to the regression coefficients obtained is as follows: Where ET0 is reference evapotranspiration (mm/day), T is average daily temperature (°C), Rh is relative humidity (%), Ws is average daily wind speed (m/s) and I sunshine duration (number of hours/day).
Statistical analysis of the data shows a close relationship between the observed and the simulated series; the   Fig. 8 Evolution of R 2 and E in terms of network architecture and number of epochs. R 2 Determination coefficient, E Nash-Sutcliffe efficiency, c number of hidden layers, n number of neurons per each hidden layer, e number of epochs determination coefficient R 2 reached 97%. Generally, all the parameters used in the models contributed significantly to estimating ET0. The results showed a confidence level of 0.05, which means that the marginal contribution of each variable is significant. They also showed that the observed F (1,445.14) was higher than the critical F (3.9). The T statistic of relative humidity is −11.40, which reflects an inverse relationship with the evapotranspiration and water requirements for cultivation. Whereas the effects of air temperature, wind speed and sunshine hours were found to be positive.
It is a natural fact that meteorological factors in general act in concert. Therefore, it is pertinent to take into account the combined influence of all the meteorological parameters on evapotranspiration. As far as the significance of individual meteorological parameters is concerned, this study revealed that the highest value of correlation coefficient was obtained for evaporation with air temperature, followed by wind speed and relative humidity. Figure 6 indicates that the observed series and the simulated series have almost the same speed, although they merge up and down several times. Nevertheless, the two series diverge occasionally, especially at the peaks of small values. Therefore, the observed values have more importance than the simulated values in some cases, and viceversa in other cases. The comparison of the ET0 predicted by MLR and the observed values shows good agreement with R 2 =0.97 (Fig. 7).
The efficiency E, R 2 , RMSE, and MSE statistics of this model for the dataset of testing phase are given in Table 2. This result shows that all these performance criteria are very satisfactory, emphasizes the factors influencing ET0 since the model considered all the variables, and indicates that the relationship between the two series is very high.

Neural networks
Using a simple neural network architecture, we obtained some very satisfactory results. Indeed, when we compared the performance criteria of each modeling phase with those of MLR, we found that the performance criteria of a singlehidden-layer architecture with four neurons was more interesting. All statistical parameters used showed that the ANN model is better than the MLR model (Table 2).
In this context, Tabari et al. (2010) have also noted from comparisons of model performances that ANN was more suitable than MLR. Also, Izadifar (2010) found that, using a single hidden layer and five neurons, the MLR model is better than the ANN model.
The results presented above are very satisfactory and we can stop with this simple architecture. In this context, Tabari et al. (2009) noted that, among several tested architectures, a single hidden layer with five neurons was the best architecture. So we can say that an ANN with only one hidden layer is enough to represent the nonlinear relationship between the climatic elements and the corresponding ET0. But, it should be noted that the advantage of the neural method lies in the possibility of making improvements in performance criteria by modifying the network architecture. Koleyni (2010) believed that the performance of a neural network is very often related to its architecture. This performance is usually determined simply through experiments due to lack of theory. The choice of the neural network  Fig. 9 Evolution of errors in term of network architecture and number of epochs. R 2 Determination coefficient, E Nash-Sutcliffe efficiency, c number of hidden layers, n number of neurons per each hidden layer, e number of epochs capacity fundamentally reflects its ability to learn and generalize. If the network model is proportionally too small, it will be unable to obtain the desired function. However, if it is too complex, it will be unable to generalize the model.
Throughout the various architectures tested, we sought to (1) maximize the determination coefficient R 2 , and (2) approximate the Nash criterion to 1. In fact, we applied a trial-and-error technique by increasing the number of neurons in the first hidden layer until further improvement ceased, and then added another hidden layer.
In fact, the improvement of model performance by adding neurons to the single hidden layer was limited to 13 neurons; thereafter it decreased. The values of MSE obtained in the test phase were 0.029, 0.019 and 0.027 (mm/day) 2 , respectively, by 12, 13 and 14 neurons. The best value obtained by the single hidden layer was bigger than 0.0047 (mm/day) 2 , which was obtained by the network architecture chosen in this study.
We found that with one hidden layer, R 2 values fluctuated, and with 2 hidden layers, R 2 values progressed in a quick and monotonous way. Moreover, the values of the Nash criterion (E) progressed significantly to reach a value of 1 at the test phase (Fig. 8).
Furthermore, the addition of other nodes may not improve model performance.
Another parameter that should absolutely be taken into consideration is the number of epochs. The different combinations show that 1,000 epochs are enough to obtain the best results. The addition of more epochs is useless and may decrease performance.
Decreasing the number of hidden layers will not automatically improve model performance. It may affect all performance criteria negatively, like the network architecture (c=2, n=4), but with the addition of neurons to each hidden layer, the rate of improvement becomes very fast.
As the two criteria approach 1, the criteria that reflect the importance of the errors between the observed and simulated values regress little by little to achieve their minimal values (Fig. 9).
Extensive test experiments were conducted in order to select the optimal network architecture. Consequently, these tests led to a network of two hidden layers, each of eight neurons.  We should mention that, as the network architecture becomes more complex, the learning process becomes more and more difficult, and the time required to perform this operation increases progressively. Therefore, modeling can take a long time and the search for a better architecture requires considerable processor time. So, the most suitable architecture in our case is a network of two hidden layers of eight neurons each.
Also, neural networks require setting up a learning rate and number of iterations. After testing different combinations, we chose a learning rate of 0.2 and a number of iterations of 1,000.
At first glance, MLR showed a remarkably satisfactory performance. Nevertheless, the neural network model outperformed MLR overall, as shown in Table 3.
Comparing the performance criteria obtained during different stages of the neural method model with those obtained by MLR for the various sets of data shows the importance of the neural network modeling. The MARE (%), i.e., the percentage of recorded errors between real and simulated values of ET0, indicates the higher performance of the neural networks over MLR. Table 3 shows the absence of overfitting because the difference between errors (MSE) at the learning and testing phases is not significant. These errors increase when moving from training phase to test phase, and then decrease at the validation phase. We should note that errors occur due to the nature of the data. Yet, in the case of MLR, the rate of errors is higher compared to the neural network model. The absence of overfitting is due mainly to the procedure adopted to avoid it and, at the same time, confirms the correct choice of neural network architecture.
In order to evaluate the correlation between the observed values of the ET0 and the simulated values, we plotted them in a graph as shown in Fig. 10. The result shows scattered points distributed statistically around the line y = x. This shows a very good resemblance that explains a high correlation coefficient between the learning, test and validation phases. We mentioned that most of the values predicted with ANN lie near the y = x line. Further, this study concludes that a combination of mean air temperature, wind speed, sunshine hours and mean relative humidity provides better performance in predicting ET0.
In addition, the statistical parameters show close resemblance between the three modeling phases. These results further confirm the high performance of the model (Table 4).
The comparison between the observed and simulated series of ET0 values reveals a high resemblance (Fig. 11).
If we compare the results obtained by ANNs in the validation phase (ET0a) with those obtained by the multiple regression method (ET0m) for the same dataset, we can see clearly that the neural networks series is a better fit (Fig. 12). The difference becomes greater at extreme values; this adds further justification to the choice of neural networks. In this context, Deswal and Pal (2008) noted that, of the two regression analysis approaches that have been used, ANNs provide better results in terms of predicting evaporation due to a higher correlation coefficient with a lower RMSE. Finally, we must confess that the performance of the models varies according to the number of inputs as well as the predicted time step. Hence, Wang et al. (2010) noted that wind velocity and relative humidity were found to improve temperature-based backpropagation accuracy when incorporated into the network input sets.
Indeed, this performance will be even better if we were interested in modeling a more extensive time step. With a simple architecture, we can obtain a very strong correlation, i.e., R 2 close to 1. Nevertheless, performance decreases when the number of the inputs is reduced.

Conclusions
The present study discusses the application and usefulness of the ANN modeling approach in predicting ET0. The results are quite encouraging and suggest the usefulness of neural network-based modeling techniques for accurate prediction of evapotranspiration as an alternative to MLR approaches, because the advantage of the neural method lies in the possibility of having improvements in the performance criteria by modifying the network architecture.
Furthermore, the results obtained confirm that neural networks have become powerful tools for modeling in many varied fields of research. They are able to construct, in a simple and effective manner, models that are precise and economical in terms of the number of parameters. Accuracy and the user's required qualities justify the choice of these approaches.
Using MLR in the simulation can result in satisfactory findings if all the conditions required for this approach are present. However, if some variables are lacking, or in order to overcome the multi-colinearity problem, the neural network method justifies its superiority in the power of prediction.
Indeed, the designed neural network model showed higher performance than MLR; the simulated series matches the observed series perfectly. However, in this research we faced great difficulty in choosing an optimal architecture. In fact, it took a long time for the trial-anderror procedure to settle on the perfect architecture.
The method adopted to overcome the overfitting problem gave satisfactory results, as well as taking account of the seasonal tendency in the ANN model. The wide range of conditions in which the input variables evolve, ensure the applicability of the models obtained in several different types of climate. The results obtained clearly justify the approach adopted in this article, and confirm its utility in a very large region, similar to the study area; arid regions are becoming larger due to the phenomenon of desertification. In addition, the Penman Monteith formula remains, until now, the formula that gives the most accurate results, and the meteorological parameters required for MLR are unavailable in most weather stations.
Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.