{'fraction_non_alphanumeric': 0.029985897721368664, 'fraction_numerical': 0.011467379202850144, 'mean_word_length': 4.761599315800727, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Remaining Useful Life Prediction for Lithium-Ion Batteries Based on CS-VMD and GRU

Accurate prediction the remaining useful life (RUL) and estimation the state of health (SOH) are critical to the management of lithium-ion batteries. In this paper, a lithium battery capacity prediction method based on cuckoo search optimization variational mode decomposition (CS-VMD) and gated recurrent unit (GRU) is proposed. Firstly, the VMD algorithm is used to divide the capacity into some intrinsic mode functions (IMFs) to reduce the impact of capacity regeneration and other situations. The number of decomposition layers and the quadratic penalty factor of VMD are optimized by the CS algorithm. Then, the GRU network is introduced to capture small changes in the capacity degradation process and perform the capacity prediction of decomposed sequence. Finally, some prediction results are integrated effectively. Based on two publicly available lithium-ion battery datasets, the model proposed in this paper can significantly reduce the complexity of the sequence and have high prediction accuracy, which is better than other prediction models. The root mean square error (RMSE) is controlled within 2%, and the maximum mean absolute error (MAE) does not exceed 2%.


I. INTRODUCTION
With the continuous progress and development of new energy technologies, lithium-ion batteries are widely used in wind power and hydropower. Compared with other chemical reactions, it has the advantages of high energy, low environmental pollution, long cycle life and no memory effect. However, the performance of the battery continues to decline in terms of capacity decay and internal resistance increase in the long-term use process, which affects the safety of the system and equipment [1]. Therefore, studying the remaining useful life (RUL) and state of health (SOH) can evaluate the state of lithium-ion batteries, which can help reduce equipment maintenance costs and improve system operation.
At present, the methods for predicting the RUL of lithiumion batteries are mainly divided into model-driven methods and data-driven methods [2], [3]. References [4], [5] evaluate the life loss through the battery operating curve. However, the measurement data usually contains a lot of noise due to the complexity of the internal reactions of the batteries. Moreover, the model-driven prediction model often requires The associate editor coordinating the review of this manuscript and approving it for publication was Nagarajan Raghavan . a lot of background knowledge and solving complex partial differential equations [6]. Considering cost and efficiency, data-driven RUL prediction models for batteries are widely used by researchers in the latest available studies.
The data-driven RUL prediction model allows exploring the relationship between the externally measured parameters and the internal state of the battery without studying the complex battery principles [7], [8], [9]. Liang et al. [10] combined the advantages of convolutional neural network (CNN) and bidirectional long short-term memory network (Bi-LSTM), and proposed a lithium-ion battery RUL prediction method that takes into account various life degradation characteristics and data timing. Zhao et al. [11] proposed an indirect prediction method for lithium batteries RUL based on whale optimization algorithm combined with extreme learning machine (ELM). However, due to some batteries have a serious capacity regeneration phenomenon, it is difficult to accurately track the battery capacity degradation trend.
In view of the capacity regeneration problem in the process of lithium battery life decline, some multi-scale decomposition methods are applied to the battery capacity sequence. Some scholars have used the method of empirical mode decomposition (EMD) combined with neural network to predict battery RUL [12]. Xiang et al. [13] proposed an improved ensemble empirical mode decomposition (EEMD) and Gaussian regression process for RUL prediction. At the same time, the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) has also been used to reduce the complexity of battery capacity sequences [14], [15].
EMD, EEMD, and CEEMDAN can adaptively divide complex battery capacity sequence data into a series of intrinsic mode functions (IMFs). However, they have not had solid theoretical foundations to support them, and still suffer from under-decomposition and over-decomposition when decomposing highly complex sequences. In order to solve this problem, scholars later proposed the variational mode decomposition (VMD) method. It solves the optimal solution of a variational model using an iterative loop to determine the center frequency and bandwidth of each component. The VMD has a good theoretical foundation and noise robustness. It has outstanding advantages in extracting complex sequential fluctuation patterns and improving the accuracy of forecasting models. Wang et al. [16] proposed a lithium battery RUL prediction method based on VMD and integrated depth model, using multilayer perceptron and LSTM to model the global degradation trend and various fluctuation components.
However, in the VMD process, the number of decomposition layers K and the secondary penalty factor α are two important parameters that decide whether the VMD decomposition will produce good results. Therefore, it is necessary to find the optimal parameter combination. Many swarm intelligence optimization algorithms are used to optimize key parameters [17], [18]. As one of the classic intelligent algorithms, the cuckoo search (CS) algorithm only contains two control parameters, population size and discovery probability, making the algorithm simpler and more efficient. In order to reduce the influence of artificially determined hyperparameters and obtain the optimal combination of these two key hyperparameters, a method combining CS and VMD is proposed. In addition, gated recurrent unit (GRU) networks have been widely used in the fields of diagnosis and prediction due to their excellent performance in recent years [19], [20].
In summary, a cuckoo search optimization variational mode decomposition and gated recurrent unit (CS-VMD-GRU) model for RUL prediction is proposed in this paper. The CS-VMD method can effectively reduce the nonsmoothness, nonlinearity and complexity of the original sequence. The GRU structure can effectively capture the small changes in the sequence change process and effectively predict the development trend of the sequence. Finally, the effectiveness of the model proposed in this paper is verified by two public datasets.
The organization and layout of the rest of our paper are as follows: Section II presents the basic theory of CS-VMD and GRU; Section III introduces the construction of the model in our paper; Section IV presents the specific experimental process and results; The last section is the conclusion of this paper and a prospect for future work.  [21]. It is a quasi-orthogonal, completely non-recursive signal decomposition method. It can decompose the signal into multiple bandwidth-limited IMFs.
Assuming that the battery capacity sequence f is composed of K finite bandwidth IMFs u k (t), where ϕ k (t) is a non-decreasing function; A k (t) represents the envelope function.
Let the center frequency of each IMF be ω k . The steps of VMD are as follows.
Firstly, the analytical signal of u k (t) is obtained by Hilbert transform.
Secondly, the bandwidth of each IMF is estimated by the following equation, The optimal solution of the above equation is solved by introducing a Lagrange multiplier operator λ (t) and a quadratic penalty factor α.
Finally, by using the multiplicative alternating direction algorithm to solve the above problem, all IMFs are obtained from equation (5).
where u n+1 k (ω) , f (ω) and λ (ω) represents the Fourier trans- Wiener filter. And ω n+1 k is updated by the follow equation (6): By introducing the VMD method to the RUL prediction of batteries, the complexity of the capacity sequence can be effectively reduced and the phenomenon of partial battery capacity regeneration can be greatly improved. Before processing the signal with VMD, the number of decomposition stages K and the secondary penalty factor α are two important parameters that determine whether the VMD can achieve good results.

B. CS ALGORITHM
CS algorithm is a new swarm intelligence algorithm based on cuckoo parasitism spawning behavior and Levy flight [22]. The search path and update position formula are as follows: where x t i is the position of the t generation bird's nest, x t+1 i is the position of the (t + 1) generation bird's nest;a represents the step size scaling factor; L(s, λ) is the Levy random walk formula, and the relationship with time obeys the Levy distribution; (x) is the gamma function.
The preferred random walk formula is as follows: where r is the uniform distribution of the scaling factor in the interval (0,1); X t j and X t k represent two random solutions of the t generation.

C. GRU METHOD
Aiming at the problems of gradient disappearance and gradient explosion in the training process of traditional recurrent neural network (RNN), some scholars proposed LSTM and GRU. It addresses the problem of vanishing and exploding gradients in RNN where network units are connected in a chain fashion. Compared with the LSTM method, GRU simplifies the complex structure and improves the computation speed [23].
It can be seen from Fig. 1 that the GRU network is a chain network model composed of a variety of different neural processing unit modules. Its basic principles are as follows: Assuming that the network input is (x 1 , x 2 , . . . , x T ), the hidden layer state is (h 1 , h 2 , . . . , h T ), and the network is at time t. The reset gate r t is mainly used to control how much the system structure ignores the state information at the previous moment.
The update gate z t is used to control how much state information of the system structure is transferred to the current state at the previous moment.
According to the weight obtained, calculate the state informationĥ t of the candidate hidden layer of the model at the current moment and the current hidden layer state information h t , then the output of the GRU unit at time t can be obtained as y t . The calculation formulas are as follows: where, b r represents the reset gate coefficient, b z represents the update gate coefficient, b 0 is the output layer coefficient, w r represents the reset gate weight matrix, w z represents the update gate weight matrix, w 0 is the weight matrix of the output layer, σ is the sigmoid activation function, and the output mapping range is (0, 1).

III. CONSTRUCTION OF CS-VMD-GRU MODEL A. OVERVIEW OF THE EXPERIMENT PROCESS
The RUL of a lithium-ion battery is the number of charge/discharges cycles the battery undergoes from new start to end of life (EOL) under certain operating conditions, as defined below.
where, n EOL represents the total number of charge and discharge cycles of the battery, and n t represents the number of charge and discharge cycles that have elapsed at the time t.
The SOH of the battery in the k − th charge and discharge cycle is the ratio of the battery capacity C k to the initial battery capacity C 0 in this cycle.
When the maximum available capacity drops to 70% of the rated capacity [24], the battery life ends.
Since the initial battery capacity of lithium batteries in each data set selected in this paper is consistent, the battery capacity data is directly used to predict healthy life. 89404 VOLUME 10, 2022

B. CS-VMD
In the current research, there is no accurate parameter adjustment method for the values of K and α. The specific process of the CS-VMD method is as follows.
Step 1: Take the two key hyperparameters K and α in the VMD algorithm as optimization objects, randomly initialize the position of the bird's nest, and assign values to the number of bird's nests n, the maximum number of iterations T and the probability of discovery p a .
Step 2: Calculate and find out the best contemporary bird's nest position X best through fitness function.
This paper selects the root mean square error (RMSE) between the true RUL value of the lithium-ion battery and the predicted RUL value as the adaptation of the CS algorithm optimization fitness function.
where, N is the predicted number of cycles,ŝ n is the predicted RUL of the lithium-ion battery, and s n is the true RUL.
Step 3: Levy flight operation is performed to update the position and state of the nest, calculate the fitness value of the updated nest position, and compare it with the fitness value of the initial nest to select a better nest position for retention.
Step 4: Randomly generate a random number r in the interval [0,1], if r > p a , then randomly improve X t+1 i according to equation (9). Otherwise, it will not change.
Step 5: The fitness values of the retained optimal nest locations are calculated and compared with the fitness values corresponding to the previous generation of nest locations, and the better nest locations are retained again.
Step 6: Judge the algorithm termination condition, if it does not meet the iteration termination condition, go back to Step3. If the termination condition is met, the optimal parameters calculated by CS are used as the final parameters of the VMD algorithm.

C. CS-VMD-GRU PREDICTION MODEL
The structure of the lithium-ion RUL prediction model proposed in this paper is shown in Fig. 2.
The specific process of the RUL prediction method is as follows.
Step 1: Obtain the data of the remaining discharge capacity of the lithium-ion batteries.
Step 2: Based on the CS-VMD model, the discharge residual capacity sequence is divided into several IMFs with relatively low complexity and relatively stable.
Step 3: Firstly, the decomposed IMF components are divided into training set and test set. Secondly, the GRU prediction models are established for each IMF component, and the hyperparameters of the GRU network are obtained through the training model. Finally, the GRU model is used to predict the i−thIMF component of the remaining discharge capacity.
Step 4: Some prediction results are integrated effectively. In the RUL prediction part, a GRU network structure is constructed. The input to each GRU network is the serial data of each IMF component of the historical capacity, and the IMF value of the next cycle is used as the output. The specific expression is as follows: Among them, imf i t is the lithium-ion historical battery capacity value of the i−th IMF component at time t, imf i t+1 is the predicted battery capacity value of the i − th component at time t+1, and w is the embedding dimension. The value of the original battery capacity sequence can be obtained by adding the values of each IMF component.
where,ŝ t+1 represents the predicted battery capacity sequence value at time t + 1. All data is normalized before being put into the GRU network to eliminate the difference between variables. The normalization method is as follows: where, zs is the normalized value, and s is the battery capacity serial value before normalization.

IV. CONSTRUCTION OF CS-VMD-GRU MODEL
A. EXPERIMENTAL DATA 1) DATABAESE-1 The first experimental dataset is from the NASA lithiumion battery dataset. The experimental data of four batteries VOLUME 10, 2022  B0005, B0006, B0007, and B0018 were selected for simulation verification [25]. The detailed specifications of the selected NASA lithium-ion batteries are shown in Table 1. Fig. 3 shows the decay curve of capacity with the number of cycles during the discharge of four lithium-ion batteries. It can be seen that different types of batteries will have capacity increase during the discharge process, indicating the nonstationarity and nonlinearity of the lithium battery capacity sequence.

2) DATABAESE-2
The second experimental dataset comes from the Center for Advanced Life Cycle Engineering (CALCE) [26]. This paper studies CS2_33, CS2_34, CS2_35, CS2_36, CS2_37, CS2_38. Fig. 4 shows the decay curve of capacity with the number of cycles during the discharge of six lithium-ion batteries. The detailed specifications of the selected CALCE lithium-ion batteries are shown in Table 2.

B. EVALUATION INDEX
In order to fully analyze the effectiveness of the selected method, six indicators were chosen to evaluate the effectiveness of the proposed prediction model.
The mean absolute error (MAE) is calculated by the following formula (22).
Root mean squared error (RMSE) response error distribution.
Mean absolute percentage error (MAPE).  Correlation index (R 2 ), adjusted correlation coefficient (Adjusted_R 2 ) and relative accuracy (RA) are used to measure the difference between the predicted RUL value and the true RUL value.
In formulas above,ŷ i and y i represents the predicted and original data, respectively.ȳ is average of the original data. n is the length of the sampled data, and p = 1.

C. COMPARISON OF DECOMPOSITION METHODS
The initial number of CS nests is 20 and the maximum number of iterations is 20. According to the equation (18), the optimal combination of the number of decomposition layers K and the secondary penalty factor α is (6, 1400). As shown in the Fig. 5, compared with the original signal, it can be seen that not only the problem of modal aliasing is avoided but also the noise component is filtered out, which fully reflects the superiority of the CS-VMD method.    5 (a) and (b) above are the timing diagrams and spectrograms of the components of the EMD method. It can be seen that IMF1 and IMF2 have different degrees of modal aliasing, and the various frequencies of the sequence are not well separated. Fig. 5 (c) and (d) also appear to be aliased in the previous components. The CS-VMD method can separate the IMF components of each frequency well, and its effect is better than that of EMD and CEEMDAN. Both EMD and CEEMDAN have different degrees of modal aliasing. Then the fixed parameter VMD method is used to decompose the original signal. The selected K and α are 4 and 2000, respectively. It can be seen that because the appropriate K is not selected, not only the under-decomposition problem occurs, but the center frequency in the original signal is also not accurately extracted.

D. EXPERIMENTAL RESULTS
In the prediction phase of the RUL of lithium-ion batteries, B0005, B0006, B0007 and B0018 batteries of NASA  dataset and CS2_33, CS2_34, CS2_35, CS2_36, CS2_37 and CS2_38 of CALCE dataset are used to train and test several related algorithm models. In this experiment, in order to experience the performance of the CS-VMD-GRU combined model, five models of LSTM, EMD-LSTM, CEEMDAN-LSTM [14], VMD-LSTM and VMD-Bi-LSTM were selected for comparison and explanation.
The GRU parameter settings used in this study are as follows. When training the GRU network model, the first 50% of the battery capacity of the CALCE and NASA datasets are used as training samples, and the last 50% of the data are used as test samples.

1) DATABASE-1
As can be seen from Fig. 6 above, due to the uncertainty of the internal chemical reaction of the lithium-ion battery during discharge, there will be an increase in capacity. The single LSTM model has a large fluctuation in the prediction of the battery capacity decay sequence. After the introduction of the EMD method, the accuracy of the sequence for some lithium-ion battery capacities has been improved. Due to the modal aliasing and the end effect of the EMD method, different degrees of error will occur when predicting the capacity attenuation sequence. In each inherent mode, there are relatively large errors in the prediction of the components, and finally the overall fitting effect of the sequence is worse. Moreover, the EMD method itself does not have a strict mathematical theoretical background and lacks rigorous mathematical derivation. The VMD algorithm determines the center frequency of each signal by iteratively solving the variational problem. The algorithm is more robust and can better decompose each IMF component. After optimizing the number of decomposition layers and the secondary penalty factor of the VMD algorithm, the signal can be decomposed more accurately, and the prediction effect is better. Fig. 7 shows the evaluation indexes of the prediction results of lithium-ion batteries under different algorithms. Among them, the smaller the RMSE, MAE and MAPE, the higher the prediction accuracy of the prediction model. It shows that the performance indicators of the combined prediction model CS-VMD-GRU are better than other methods, with high accuracy. The maximum RMSE is within 3%, the maximum MAE is within 2%, and the maximum MAPE is within 6%. Obviously, it has a relatively high prediction accuracy. The average RMSE predicted by traditional LSTM is 0.0347, and the average RMSE of CS-VMD-GRU proposed in this paper is 0.0142, which reduces the average RMSE by 59.08%. The average MAE predicted by traditional LSTM is 0.0277, the average MAE of CS-VMD-GRU proposed in this paper is 0.0112, and the average MAE is reduced by 59.57%. The average MAPE predicted by traditional LSTM is 0.1352, and the average MAE of CS-VMD-GRU proposed in this paper is 0.0282, which reduces the average MAE by 79.14%.
The capacity value at which the capacity decays to 70% of the initial capacity is used as the life threshold, and then the number of usable cycles for the lithium-ion batteries to reach the EOL is predicted.
where RUL er is the RUL prediction error, RUL pr is the RUL prediction value, RUL tr is the RUL true value, and P er is the relative error of the lithium-ion battery prediction.
According to the analysis in Table 3 above, it can be seen that the prediction model presented in this paper has the smallest average prediction error for lithium-ion batteries, and there will be no major fluctuations. For example, when CS2_37 used EMD-LSTM and CEEMDAN-LSTM for prediction, their relative errors both exceeded 10%, and when CS2_38 used VMD-Bi-LSTM model for RUL prediction, their relative errors also exceeded 10%. In contrast, the average relative error of the CS-VMD-GRU model proposed in this paper is the smallest, reflecting the better adaptability and robustness of this method.

2) DATABASE-2
As can be seen from the above Fig. 8, the method in this paper also has a comparatively predictive effect on NASA dataset. In view of the phenomenon of battery capacity regeneration, VOLUME 10, 2022 the capacity decay sequence of lithium-ion batteries can be better predicted.
The reference battery capacity reaches the failure threshold as the standard for the end of battery life. The failure threshold is set to 1.4Ah, but for the B0007 battery, the remaining capacity has not reached 1.4Ah at the end of the test experiment, so the failure threshold is set to 1.42Ah. The battery failure threshold and actual life are shown in Table 4 below: According to the analysis in Table 5 above, it can be seen that the new prediction model presented in this paper has the smallest error in the prediction of lithium-ion batteries, and there will be no major fluctuations. For example, when B0005 uses EMD-LSTM and CEEMDAN-LSTM for prediction, the relative error reaches 20%. This is because the RUL of the lithium-ion battery fluctuates significantly near the life threshold, causing the prediction to be inaccurate. In comparison, the average relative error of the CS-VMD-GRU model proposed in this paper is the smallest, which reflects the better adaptability and robustness of the algorithm, and has a certain improvement in accuracy compared to other models. Table 6 is the results of each index of the method proposed in this paper on the two datasets. For the same sample data, the CS-VMD-GRU combination algorithm is compared with several commonly used algorithms. It can be seen that the prediction accuracy of several other models decreases when the training data is too small, but the combined algorithm proposed in this paper still has a high prediction accuracy, which reflects that the combined prediction algorithm is sufficiently trained on the data to improve the prediction accuracy and reduce the strong dependence on the data. Moreover, due to the characteristics of GRU own network, it also reduces the computational effort without reducing the prediction accuracy compared with LSTM network.

V. CONCLUSION
Lithium-ion battery capacity degradation is a very significant issue in practical applications. The prediction of battery RUL is closely related to the safety of timely maintenance and replacement of electric equipment systems. In this paper, a lithium-ion battery RUL prediction method based on CS-VMD-GRU is proposed. This method adopts the idea of combining time-frequency analysis and neural network prediction to predict the decay sequence of lithium-ion battery capacity. Firstly, the complex battery discharge capacity decay sequence is divided by CS-VMD to obtain each IMF component. By dividing the training set and test set of different proportions, the GRU model is used to train and predict each IMF component, and finally the sub-sequences obtained by the prediction are effectively integrated to obtain the final battery RUL.
Experimental results based on NASA lithium-ion battery dataset and CALCE lithium-ion battery dataset show that CS-VMD can significantly reduce the complexity of the sequence and reduce the modal aliasing phenomenon of each IMF component. Compared with LSTM, EMD-LSTM, CEEMDAN-LSTM, VMD-LSTM, VMD-Bi-LSTM methods, the method presented in this paper has very high prediction accuracy, the maximum MAE of prediction does not exceed 2%, and the RMSE is controlled within 2%. Because VMD needs to iteratively solve the variational equation, the training time of the model proposed in this paper is increased compared with other methods. In the following work, we will try to use the improved empirical wavelet transform to decompose the sequence, which can improve the prediction accuracy at the same time and save model training time.
GUORONG DING received the bachelor's degree in statistics from the Wuhan University of Science and Technology, Wuhan, China. He is currently working as a Lecturer with the College of Science, Wuhan University of Science and Technology.
WENBO WANG received the Ph.D. degree in computational mathematics from Wuhan University, Wuhan, China. He is currently working as a Professor with the College of Science, Wuhan University of Science and Technology.
TING ZHU received the bachelor's degree in statistics from Huanggang Normal University, Huanggang, China. She is currently working as a Lecturer with the College of Science, Wuhan University of Science and Technology. Her research interests include application of wind speed prediction and remaining useful life prediction. VOLUME 10, 2022