{'fraction_non_alphanumeric': 0.028900993271387374, 'fraction_numerical': 0.008458827298942647, 'mean_word_length': 4.567606136282555, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Comparative Study of Short-Term Forecasting Methods for Soybean Oil Futures Based on LSTM, SVR, ES and Wavelet Transformation

Short-term forecasting of futures market is valuable and is also a technical challenge. In this paper, a hybrid approach for soybean oil futures price forecasting is proposed based on time-series analysis methods. The method combines wavelet transformation and exponential smoothing so that the characteristics of the time series can be captured at different time scales, and forecasting based on exponential smoothing is applied at each time scale. A comparative case study is then conducted that compares the proposed method with other three methods which are an RNN network with Long Short-Term Memory units, a Support-Vector Regression model, and an Exponential Smoothing model without wavelet decomposition to the time series. It could be concluded that the forecasting error performance of ES and Wavelet-ES was better than LSTM and SVR, and the Wavelet-ES achieved the best results for the direction forecasting. The case study provides valuable reference for application of short-term futures price forecasting.


Introduction
Soybean is an important kind of plant that has been widely grown in several countries. Soybean oil can be extracted from soybeans and is one of the mostly used cooking oil all over the world. In recent years, the prices of soybean oil fluctuate dramatically, which causes significant influence on the consumers' daily lives. Soybean oil futures contracts are a kind of financial instruments that allow trading of soybean oil at a predetermined price in the future. The nature of the contracts makes them possible to be used as a potential indication of the ongoing trends of the product. Therefore, the research on the forecasting of soybean oil futures prices can be significantly beneficial to the market participants as well as policy makers.
The forecasting of commodity futures prices has long been studied, such as crude oil and gold. However, soybean oil futures prices forecasting has not been extensively covered by researchers. The forecasting can be achieved by investigating the relationship between the futures prices and other related factors. Wang and Liu analyzed the soybean futures prices according to its relation to the market spot prices using the VAR model [1] . Wang also investigated Sino-US soybean oil futures based on the relation of the same futures contract from different countries' futures exchanges [2] . On the other hand, soybean oil futures prices can also be analyzed using Time-Series (TS) analysis methods [3] .
Conventional TS forecasting methods have been widely used in the forecasting of TS data, such as Autoregressive Integrated Moving Average (ARIMA) [1] [5] , and Seasonal ARIMA (SARIMA) [6] [7] . However, due to the linear nature of those methods, the TS data are assumed to be stationary. The limited capability of handling non-linear and non-stationary data makes it not suitable for financial instrument prices.
In order to deal with non-linear and non-stationary TS data, Artificial Intelligence (AI) and Machine Learning (ML) methods are widely adopted in financial instrument price forecasting, such as deep learning with Recurrent Neural Network (RNN) [8][9] [10] , and Support-Vector Regression (SVR) [11] [12] .
Exponential Smoothing (ES) is a method for smoothing TS data using exponential window function, in which exponential functions are used to assign exponentially decreasing weights over time. A number of variants can be applied that are capable of handling TS data that can be either stationary or not [13] [14] .
Financial instrument prices, especially soybean oil prices, can be influenced by multiple factors such as soybean production calendar, seasonal consumption, weather, trading policies, etc. Different factors change at different time scales. The Wavelet Transformation (WT) technique can be used to decompose the TS into sub-TSs of different frequencies so that forecasting techniques can be applied to each individual frequency [15][16] [17] . This can be beneficial to capture the characteristics of the TS in terms of its influence factors.
In this paper, a hybrid approach Wavelet-ES for soybean oil futures price forecasting is proposed based on TS analysis methods. The method combines WT and ES so that the characteristics of the TS can be captured at different time scales, and forecasting based on ES is applied at each time scale. A comparative study is then conducted that compares the proposed method with comparable methods, namely an RNN network with Long Short-Term Memory (LSTM) units, a Support-Vector Regression (SVR) model, and an ES model without WT decomposition to the TS.
The rest of the paper is organized as follow. Section 2 introduces the three comparing methods as well as the proposed method. Experiment results are presented in Section 3. Section 4 presents the concluding remarks.

Methods
In this section, the three comparing methods and the proposed method are introduced.

The LSTM Model
An RNN network with LSTM units is adopted for the forecasting of soybean oil futures prices. RNN is a kind of Artificial Neural Network (ANN) in which units contain recurrent connections. This kind of connections can be used to pass information from a time step to the next.
LSTM is an improved RNN architecture which can be used to solve the vanishing gradient and exploding gradient problems.
A LSTM network can be facilitated for the classification, processing, and prediction of TS data, since it is capable of capturing the lags of unknown duration between important events.
A typical LSTM unit consists a cell, an input gate, an output gate and a forget gate. The cell remembers values over arbitrary time intervals and the three gates regulate the flow of information into and out of the cell [18] , as shown in Fig. 1. The outputs of the cell and gates are calculated as follows: where matrices W and U contain the weights of the input and recurrent connections, respectively, the operator ∘ denotes the Hadamard product (element-wise product), and the subscript t indexes the time step of the TS.

The SVR Model
The SVR model is based on the Support Vector Machine (SVM) model, which is a supervised learning method in the ML field. It has been widely used for data classification. A data point is considered as a vector. The algorithm aims to find a hyperplane that can separate data points of different classes with a maximized margin.
If the data points are not linear separable in its original space, they are mapped to a higherdimensional space using a kernel function. This allows effective handling of complex problems with non-linear classification.
When solving the regression problem, instead of finding a maximum-margin hyperplane to separate the data points in the SVM model, the SVR model aims to find a hyperplane that minimize the distances to the data points.
Training the SVR model means solving where x i is a training sample with target value y i , the inner product plus intercept 〈 , 〉 is the prediction for that sample, and is a threshold.
The SVR model can also be used for TS forecasting if recent k data points {x t-k+1 , x t-k+2 , …, x t } in the past are used as the input data to forecast the next coming data x t+1 .

The ES Model
The ES model is a smoothing technique for TS data that smooth the current data point using past data points of a time window. Unlike the conventional Moving Average (MA) model that assigns equal weights to past data points, the ES model assigns more weights to more recent data points using an exponential function. This can be beneficial for handling TS data that are non-stationary and non-linear [17] .
The ES model can be applied in a wide variety in terms of its trend and seasonal components. Both components can be classified as none, additive, and multiplicative according to the nature of the data. In this paper, an ES model with additive trend and additive multiplicative is adopted, namely the Holt-Winters' additive method [19] [20] . The forecasting is achieved recursively as follows:

The Proposed Wavelet-ES Model
A hybrid approach is proposed that combines WT and ES models. The WT model can be used to decompose signals into different frequencies and is capable of capturing both frequency and location information while the conventional Fourier Transform (FT) model can only capture frequency information. When used in TS analysis, it is especially useful to handle non-stationary TSs and is more flexible than Short-Time Fourier Transform (STFT) in terms of handling the time window.
The time-scale WT of a continuous TS f(t) is denoted by [21] : where , * is the mother wavelet conjugate. The wavelet coefficients , are obtained by continuously varying the scale parameter and the position parameter in order to select the different portions of the signal and analyze the different scale variations.
In a discrete-time signal processing scenario, a dyadic grid is employed, where the mother wavelet is scaled by power two ( 2 ) and translated by an integer ( 2 ). A Discrete Wavelet Transform (DWT) is expressed by the following equation.
The DWT coefficients are obtained from the following expression.
In order to reconstruct the original TS, an Inverse Discrete Wavelet Transform (IDWT) is calculated from the wavelet coefficients , using the following formula.
Soybean oil futures price can be affected by a composition of different influencing factors, each of which can affect the price at a different time scale. Past data points in the TS are first decomposed using DWT into sub-TSs of different frequencies to capture different influencing factors at different time scales. A single-level DWT decomposes a TS into an approximation sub-TS and a detail sub-TS. Multilevel DWT can be achieved by further applying the decomposition to the approximation sub-TS at each subsequent level. A n-level DWT generates n+1 sub-TSs A n , D n , D n-1 , …, D 2 , D 1 , where A n is the approximation sub-TS generated in the n-th level, and D k ( ∈ 1, 2, … , ) is the detail sub-TS generated in the k-th level.
After the decomposition of the TS, forecasting techniques can be applied to each individual frequency sub-TS. In the proposed approach, a simple ES model without trend or seasonal components is applied to each sub-TS.

Experiment
Experiments are conducted for a comparative analysis that compares the proposed method with comparable methods.  The characteristics of the TS is measured. The overall mean of the TS is 7719.76, and the overall standard deviation is 1789.66. The rolling mean and standard deviation of the TS are shown in Fig. 3. From Fig. 3(a) we can see that the rolling standard deviation of the original TS is relevantly stable while the rolling mean is rather dynamic. The result of Dickey-Fuller (DF) test suggests that the original TS is not stationary. However, the original TS can be made stationary by applying first-order differencing. The results can be seen in Fig. 3(b).  Although the TS can be made stationary easily by applying first-order differencing, we can see from Fig. 3(b) that the statistical properties of the transformed TS is not always stable during some period of time, especially when the market volatility is rather high. Therefore, in order to achieve automatic online forecasting of soybean oil future prices, sophisticated methods that are capable of handling nonstationary data are better to be adopted for the forecasting task.

Evaluation Metric
The Root-Mean-Square Error (RMSE) metric is selected for the evaluation of the forecasting results. RMSE is a widely used metric that is used to measure the difference between predicted values computed by a model and the observed values. The RMSE of a predicted TS , , … , and a observed TS , , … , is calculated as follows:

Experiment Results
A comparative study is conducted to the TS that compares the proposed method Wavelet-ES with comparable methods, namely LSTM, SVR, ES. An out-of-sample walk-forward validation test is conducted to evaluate the four models. For each time step ∈ 1000, 1001, … , 1 , its recent 1000 data points {x t-999 , x t-998 , …, x t } in the past are used as the training data to forecast the data point x t+1 that comes next. For SVR, ES, and Wavelet-ES algorithms, a forecasted value is acquired for every time step and compared with the observed value. However, due to lengthy training time, for the LSTM algorithm the test is run for every 10 time steps. The forecasted values are compared with the observed ones for the evaluation of the models.
For the proposed Wavelet-ES model, a 2-level DWT is applied to the original TS. The decomposition result for the earliest training data {x 1 , x 2 , …, x 1000 } is demonstrated in Fig. 4. From the figure we can see that the original TS is decomposed into three sub-TSs with different frequencies.  The evaluating results of the models are shown in Table 1 and the forecasted price movements are shown in Fig. 5. From the table it can be observed that ES and Wavelet-ES achieve better results than LSTM and SVR. This demonstrate the strong capability of ES for handling non-stationary data in soybean oil price forecasting. Especially, the RMSE is even lower when the WT is adopted for the decomposition of the TS into different frequencies. On the other hand, the LSTM and SVR models are less effective than the other two. This might be an indication that ML algorithms are less capable for soybean oil futures forecasting than TS smoothing models due to the highly dynamic nature of the movements of the futures prices.  The forecasting effectiveness of the models in different time periods is evaluated. The absolute differences between the forecasted and the actual prices are illustrated over time in Fig. 6. It can be seen from the figure that the forecasted period from 2010 to 2015 can be roughly divided into two parts. Period 1 is from late 2010 to the middle of 2012, in which the price movement is relevantly stable. Period 2 is from the middle of 2012 to the middle of 2015, in which the prices go down dramatically with a strong movement trend. It can be found that all four models achieve good results when the prices are relevantly stable while ES and the proposed Wavelet-ES are even better in Period 1. However, during Period 2 in which the price movement is more dynamic, the absolute differences for LSTM and SVR are increased dramatically which indicates the low performances of them in this period. On the contrast, ES and Wavelet-ES achieve even lower differences in this period, which indicates their effectiveness to follow trends.  Fig.6. Forecasting differences of the models over time The capability of forecasting the price movement directions of the models is shown in Table 2. It can be seen in the table that all four models achieve correctly forecasted movement directions around 50% for the next day. This is due to the random walk nature of the short-term price movement. Among the four models, the proposed Wavelet-ES achieves the best results for the direction forecasting. This demonstrates its effectiveness to capture short-term price movements. The efficiency of the four methods to train a model for the TS forecasting is also evaluated and the result is shown in Table 3. From the table we can see that training an SVR model takes the shortest time. Training the proposed Wavelet-ES model achieves the second place and is also relevantly fast. On the other hand, the training time for LSTM is the longest among the four models.