{'fraction_non_alphanumeric': 0.026163868439171654, 'fraction_numerical': 0.011281182140776442, 'mean_word_length': 5.057747834456208, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Machine Learning and Statistical Techniques for Daily Wind Energy Prediction

This paper presents the development of wind energy prediction models for the Nala Danavi wind farm in Sri Lanka by using machine learning and statistical techniques. Wind speed and ambient temperature were used as the input variables in modeling while the daily wind energy production was the output variable. Correlation between the wind energy and each weather index was investigated using the Pearson’s and Spearman’s correlation coefficients and it was found that daily wind energy output is positively correlated with both daily averaged input variables. Statistical prediction models of Multiple Linear Regression (MLR) and Power Regression (PR) and the machine learning techniques of Support Vector Regression (SVR), Gaussian Process Regression (GPR), Feed Forward Backpropagation Neural Network (FFBPNN), CascadeForward Backpropagation Neural Network (CFBPNN) and Recurrent Neural Network (RNN) were developed. The accuracy of the prediction models was measured in terms of the coefficient of determination, Bias, Percent Root mean square error (RMSE)Bias, and Nash-Sutcliffe Efficiency (NSE). Results of the performance evaluation indicated that all the models are highly accurate while the FFBPNN-based model demonstrates outstanding performance with very low error. Such prediction models are highly important for a country like Sri Lanka whose power generation mainly depends on imported coal followed by hydropower and expanding the on-shore and off-shore wind farms gradually in many potential locations scattered over the country. Received: 02 July 2021 Accepted: 08 Dec 2021


INTRODUCTION
Wind is a major renewable energy source, which generates electricity without combustion. However, the uncertain behavior of wind increases the cost of energy and limits the advantages of this pollution-free technology. Ability to predict wind energy brings multiple benefits in operating wind farms effectively in terms of regulating, scheduling and load balancing. Due to its impact on the security, reliability, and operational performance of power grids, wind energy prediction is of extreme significance to the energy sector. Further, it is useful in strategic planning, taking preventive measures, and optimizing the operation, which ultimately maximizes the revenue. Hence, researchers worldwide have suggested numerous machine learning, statistical, time series and hybrid models for wind energy prediction [1][2].
The most frequently reported wind power forecasting models are Auto Regressive Moving Average (ARMA) models, Artificial Neural Networks (ANNs), and Adaptive Neuro-Fuzzy Inference Systems (ANFIS) [3]. ARMA is a time series model, which performs linear mapping between the inputs and outputs, while the other two methods perform a nonlinear mapping [4]. Improved versions of ARMA such as Auto Regressive Integrated Moving Average (ARIMA) and Seasonal ARIMA (SARIMA) were also widely used in wind energy prediction [5][6]. Statistical prediction models of Multiple Linear Regression (MLR) and Power Regression (PR) were developed based on historical data of a wind farm such as wind speed, power output, wind direction and ambient temperature [7]. Regression-based Machine learning techniques viz. Support Vector Regression (SVR), Nearest Neighbor Regression, Multi-Kernel Robust Ridge Regression [8], Local Linear Regression [9], and Gaussian Process Regression (GPR) were widely used for developing wind energy prediction models [10][11][12][13]. ANNs, recurrent neural networks, and Convolutional Networks are among the neural networks used to model nonlinear systems for wind energy prediction [14].
In keeping with global commitments towards improving renewable energy contribution to 70% by the year 2030, Sri Lanka attach much emphasis on bolstering future electricity demand with power generated from solar, wind, and hydropower sources to good effect. However, operation of wind farms in Sri Lanka is still in the initial stage and only limited research has been conducted to predict their wind energy generation. For example, an adaptive filter was used for forecasting power fluctuations of a wind turbine installed in Kalpitiya area of the north-western region in Sri Lanka, which predicted its power generation with 5.07% Root Mean Square Error (RMSE) of its mean value, [15]. Another recent study, [7], has reported that ANN, MLR, and PR techniques are good at producing noteworthy performances with Pearson's correlation coefficient (R) values of 0.97, 0.95, and 0.94 respectively, to predict wind power at Pawan Danavi wind farm, Sri Lanka. Nevertheless, these studies used only periodically averaged monthly data over the past few years and therefore the applicability of prediction models presented in them remains to be verified for daily averaged data that should be considered for more precise and updated forecasts in the Sri Lankan context where the reliance on long-term data is questionable. In order to address this aspect comprehensively with multiple techniques, this study focused on developing models to forecast daily wind power generation at a major wind farm (Nala Danavi) in Sri Lanka based on daily averaged data. In an era where both on-shore and off-shore wind farms are being expanded in potential locations over the country to reduce the expenses on importing coal and the dependency on uncertain hydropower generation, accurate prediction of wind energy generation is highly important in numerous aspects. Though researchers have recommended various machine learning or statistical techniques for the accurate prediction of energy generation of wind farms located all over the world, no single method could be generalized for all the case studies. As such, some commonly used prediction techniques are applied in this research to find the most accurate model from among them.
More specifically, this paper presents the use of two statistical techniques in MLR and PR and five machine learning techniques: SVR, GPR, Feed Forward Backpropagation Neural Network (FFBPNN), Cascade-Forward Backpropagation Neural Network (CFBPNN) and Recurrent Neural Network (RNN) on daily averaged weather data at Nala Danavi area for the prediction of daily energy generation. Section 2 elaborates wind farm data, prediction models, and performance evaluation criteria. The performance of the models and the results obtained are presented in Section 3 with a discussion on related work and the conclusions are summarized in Section 4.

Wind Farm Data
Nala Danavi wind farm (08°05′23″N 79°42′33″E) has established itself as one of the major onshore installations on the western coast of Sri Lanka within the Puttalam district, which could generate up to 6 850 KW rated power by six Gamesa G58 turbines. Each turbine has cut-in wind speed of 3.0 m/s and cut-out speed of 20.0 m/s. As per the data analyzed over the 5-year period from 2016 to 2020, the daily production of wind energy varies between 0.3 MWh and 107.8 MWh having a mean of about 33 MWh and a standard deviation of 31.8 MWh. Figure 1 shows the variation of daily wind energy in the year 2019. The annual pattern of energy production is characterized by mostly optimum values within 50-100 MWh with some drops to within 20-40 MWh, during the generally warmer months of the year from May to September. During the other months, i.e. from October to April of the ensuing year, the daily energy generation is less than 20 MWh. Figure 2 shows the relationship between wind energy and the weather indices of wind speed and ambient temperature. A linear correlation with R² = 0.7969 can be inferred between the daily wind energy output and the average daily wind speed when the latter varies between 3-12 m/s. Nevertheless, the relationship of the daily wind energy output with the ambient temperature, which varies within 25 o C-53 o C, is not as linear as that with the wind speed due to relatively wide scattering of some data points around the line of best fit resulting R² = 0.5696.
In order to quantify the seemingly linear relationships mentioned above and the presumably pairwise correlations among the wind energy, wind speed, and the ambient temperature, the Pearson's correlation (R) and the Spearman's correlation (ρ) were calculated (Table 1) for each pair of these variables. As was suggested by the scatter plots of Figure 2, the strongest correlation was found between the wind speed and wind energy substantiated by the higher values of both correlation coefficients (> 0.80). Similarly, a significant correlation is also indicated between the wind energy and the ambient temperature with both coefficients being 0.75. Coherent with the higher energy output during the warmer months of May to September noticed in Figure 1, wind speed and ambient temperature are also strongly correlated as evident from the higher values of both coefficients. Hence, it can be deduced that all three variables are intercorrelated and the daily wind energy depends on both wind speed and the ambient temperature prevalent on the day.  The optimum wind power can be produced if the rotor blades capture the direction of the wind. At Nala Danavi wind towers, all rotors are rotated according to the wind direction with angled blades ensuring optimum energy generation at any given time. All wind data were separated into sixteen categories based on the direction of the wind and compiled with the percentage of data in each direction, as illustrated in the wind rose diagram ( Figure 3).

Figure 3. Wind rose diagram
It could be discovered that 16-20% of wind blown in the direction of southwest have made the highest single contribution from among all directions, followed by 12-16% blown in the direction of southsouthwest. The lowest percentage up to 4% of wind data were found towards east and between north and west.

Prediction Models
Daily wind energy prediction models were developed by using averaged daily wind speed and ambient temperature as inputs. Modelling was performed in MATLAB Version R2018a (Figure4). The performance of the models was evaluated in terms of the coefficient of determination (R 2 ), Bias, Percent Bias (PBias), RMSE, RMSE-observations standard deviation ratio (RSR), and Nash-Sutcliffe efficiency (NSE). Mathematical basis behind each regression method can be explained as follows.
where, β 0 is a constant, β 1 to β n are the coefficients relating n number of explanatory variables to the variables of interest and e_ is the error term. Power regression (PR) develops a relationship by modeling the nonlinearity of data. The dependent variable is expressed proportional to the product of powers of the independent variables as follows [17] = 1 2 … where, n is the number of observations and a,b,c,…,p are constants. Support Vector Regression (SVR) is a relatively new and promising supervised learning technique. For a one-dimensional problem, SVR expresses the continuous-valued function as [18] = ∑ + =1 where, w is the slope and b is the intercept. For multidimensional data, x is augmented by one and b is included in the w vector to express the multivariate regression as follows where, T is the transpose of the vector. A Gaussian process is a stochastic process described by its mean function, which is a real-valued function, and its covariance function which models the joint variability of the Gaussian process random variables [19]. MATLAB enables the application of Kernel functions of Rational Quadratic, Exponential, Squared Exponential, and Matern 5/2.
FFBPNN is a layered feed-forward ANN in which the back propagation algorithm, which uses supervised learning, is applied [20]. The backpropagation neural network architecture is a hierarchical design consisting of fully interconnected processing layers including one or more intermediate hidden layers. However, in FFBPNN the number of layers is kept low to reduce the training time. The connection formed between input and output through an activation function in the hidden layer is indirect and nonlinear. Further, the signals are sent forward and the errors are propagated backwards. After finding the error, the weights are adjusted using the gradient descendent method [21]. In contrast, CFBPNN forms a network with direct connection between the input and the output layers besides the indirect connection [22]. These additional connections improve the modeling speed. RNN architectures range from fully interconnected to partially connected networks. In fully connected networks, each node receives inputs from all other nodes and sends feedback to the node itself [23]. RNNs use feedback loops to process a sequence of data allowing information to persist. Therefore, RNNs outperform many other algorithms in forming a much deeper understanding of a sequence.

Performance Evaluation Criteria
The R 2 describes the degree of collinearity between observed and predicted data. It indicates the proportion of the variance in observed data explained by the model. In the range from 0 to 1, higher R 2 values indicate less error variance. Typically, a prediction model with R 2 ≥ 0.5 is considered acceptable [ where, and represent observed and predicted energy data. Although it is widely used for the evaluation of prediction models, it is oversensitive to outliers. On the other hand, it is insensitive to additive and proportional differences between the observed and predicted data.
Bias is the mean error, which signifies whether the method over-estimates or under-estimates the forecast variable. However, it does not indicate the level of skill of the forecast method PBias is the deviation of data being evaluated, which is expressed as a percentage. It measures whether the average tendency of the predicted data is larger or smaller than their observed counterparts. PBias = 0 indicates a perfect prediction model. Positive PBIAS values indicate model overestimation while negative PBias values indicate underestimation RMSE is expressed in the same units as the forecast variable. RMSE= 0 indicates a perfect fit RSR standardizes RMSE using the standard deviation of the observations and combining an error index with additional information. It is the ratio of the RMSE and standard deviation of observed data. RSR incorporates the benefits of error index statistics and includes a normalization factor. It is a positive value with optimal value of 0, which indicates zero RMSE or residual variation implying a perfect prediction = .
(9) NSE is a normalized statistic, which indicates how well the plot of observed versus predicted values fits [25]. It is the relative magnitude of the residual variance (noise) compared to the measured data variance (information). NSE is in the range between −∞ and 1. Values between 0 and 1 are considered as acceptable levels of performance and NSE = 1 is the optimal value. NSE ≤ 0 indicates unacceptable performance as the mean observed value is a better predictor than the value given by the model (predicted value) ] .

Results
The daily wind energy predicted by the two statistical models of MLR and PR and the five machine learning models of SVR, GPR, FFBPNN, CFBPNN and RNN were plotted against the actual energy within the range of 0 to 120 MWh as shown in Figure 4. In order to compare the performance of these models and to identify the differences of predictions generated by them, the statistical measures of R 2 , Bias, PBias, RMSE, RSR, and the NSE were calculated for each model and summarized into Table 2. In general, all the models have shown their dexterity with reasonable accuracy in predicting the daily wind energy output as indicated by the higher values of the coefficient of determination (R 2 ≥ 0.82). Further, the models can be distinguished based on the degree of overestimation or the underestimation of daily energy predicted by them, as interpreted in Section 2.3.

Discussion
The outstanding performance of the FFBPNN may be due to its capability in realizing arbitrary complex nonlinear mapping between input and output data. Further, its self-learning ability considers both inputs and predicted outputs in the training process, enabling the adjustment of the network parameters based on the errors. In addition to the performance comparison among the models developed in this research, similar research studies were also explored ( Table 3). It could be observed that the accuracy of the proposed prediction models is comparable. Though some prediction models were developed for another wind farm in Sri Lanka, forecasting was possible only on the monthly scale [7]. More importantly, no previous research has been conducted on the Nala Danawi wind farm.

CONCLUSIONS
Accurate prediction of daily wind energy generation at Nala Danavi wind farm in Sri Lanka was modelled in this research by applying two statistical techniques and five machine learning techniques. Data comprised of daily wind energy generation, wind speed, ambient temperature, and the direction of the wind though the latter is automatically adjusted to produce the optimum energy continuously. The possible correlations among these variables were explored first using the Spearman's and Pearson's correlation coefficients. Based on the pairwise correlation coefficients it could be concluded that strong intercorrelations exist among the above variables and the daily wind energy output is proportional to both wind speed and ambient temperature. Data further revealed that the wind at Nala Danavi blows mostly in the directions of southwest and south-southwest. The windy weather prevalent during the warmer months of the year accounts for higher wind power output.
The efficacy of the seven models for prediction purpose was evaluated using the six statistical measures viz. R 2 , Bias , PBias , RMSE, RSR and NSE. Generally speaking, all the models were found capable of predicting the daily wind energy generation at Nala Danavi using weather data. There was enough evidence to conclude that the model based on the machine learning technique FFBPNN is the most suitable instrument to predict the daily wind energy output, provided the availability of data on wind speed and ambient temperature. Though reasonably accurate, the models based on the techniques CFBPNN and the RNN tend to underestimate the predicted values while the regression-based models of SVR and PR tend to overestimate the predictions. Thus, this study has presented not only a viable set of models for the prediction of daily wind energy output at Nala Danavi wind farm in Sri Lanka but also temporal information on optimum wind power generation, which can be appropriately utilized to control the integration of daily wind power into the national grid and thereby preserve much valued other renewable energy sources like hydro power and more expensive thermal power.