{'fraction_non_alphanumeric': 0.02056207098223905, 'fraction_numerical': 0.012435743528180504, 'mean_word_length': 4.776671408250356, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Gabor-GLCM-Based Texture Feature Extraction Using Flame Image to Predict the O2 Content and NOx

Flame image feature extraction is the basis for boiler combustion monitoring and control. The flame video images of recent research are mainly derived from experimental burners in the laboratory, and few pay attention to the flame images in industrial boilers. The actual industrial boiler flame images differ significantly from the laboratory flame images. Additionally, certain flame image features cannot be captured in the laboratory owing to the limitations of the camera installations. Therefore, a flame image texture feature extraction algorithm based on an industrial boiler is proposed in this paper. The texture features were enhanced using a Gabor filter for the RGB channels of the flame images, and then, the statistics of the texture features were scalarized by a gray-level co-occurrence matrix (GLCM). The data were filtered and downscaled by a data compressor consisting of Gaussian-weighted mean and principal component analysis (PCA) to obtain eight key variables. The extracted eight variables were verified to be effective in characterizing the O2 and NOx contents of flue gas using the mutual information method. The combustion process regression model was constructed using a gated recurrent unit (GRU) on the 8 h combustion data of the boiler, and the predicted mean absolute percentage error (MAPE) for O2 and NOx content in the test set reached 7.5 and 10.2%, respectively. Compared to the conventional methods of direct PCA on images and GLCM plus PCA on images, the MAPE for O2 content prediction was reduced by 12.3 and 7.3%, and the MAPE for NOx content prediction was reduced by 10.5 and 6.1%, respectively. The advantage of the new flame feature based on Gabor-GLCM is suitable for the subsequent analysis and control of an industrial combustion system.


INTRODUCTION
Flame characterization techniques have been widely used to monitor modern industrial boiler combustion processes. 1 The use of cameras to capture information from the boiler helps to analyze the state of combustion. 2,3 A qualitative and quantitative description of the combustion process is key to monitoring the combustion of boilers, improving the combustion efficiency, and reducing pollutant emissions. The image of the boiler flame combustion captured by a chargecoupled device (CCD) camera contains a large amount of redundant image information. Advanced image processing techniques for flame images can greatly reduce irrelevant information and retain sensitive information about one or more characteristics of the flame related to operating load, control parameters associated with combustion states, pollutant emissions, and so on. 4 Chen et al. 5 used the Gaussian regression method to combine flame image information with boiler parameters, and more accurate results were obtained than using the neural network method. In another paper of theirs, 7 PCA was used to extract the characteristics of flame images in RGB channels and combine them with the oxygen content at the boiler outlet. A minimum variance cascade control system based on flame characteristics and oxygen content was constructed, which can improve the automatic control ability of the combustion system. They used combustion image information as the input for boiler combustion status monitoring or control and achieved some results. But the authors performed principal component analysis (PCA) directly on the original flame image without extracting quantifiable flame characteristics. Toth et al. 6 extracted the color features of the RGB channel of a flame image and applied a deep neural network combined with conventional operating parameters to predict the thermal output of a 3 MW chain biomass boiler. Xiangyu et al. 8 obtained the color characteristics of the radiation spectrum under different channels using flame images, and the combustion temperature of the flame in the furnace was calculated. Zhang et al. 9 used an improved colorimetric method based on the nonlinear least-square method to obtain the spectral characteristics of the flame image; the changes in flame temperature and flicker frequency under different combustion conditions were obtained through calculation, which was applied to the combustion control. The works in the literature make full use of flame color information. Although the flame color characteristics can well-represent the flame temperature and combustion adequacy, the pure flame color characteristics are limited to describing other combustion states of the flame. They also can not visually reflect the chemical reaction state of pulverized coal combustion in the furnace. No single flame image feature, including flame color, can completely describe the flame combustion state accurately. Bai et al. 10 extracted the flame color and texture features and derived the image features under different combustion conditions using PCA and neural networks. However, the authors directly applied the gray-level co-occurrence matrix (GLCM) method to extract features from the original image, causing the image to lose abundant vector information before PCA was applied. Sun et al. 11 combined flame color characteristics, geometric characteristics, and brightness characteristics to propose quantitative indicators for flame stability judgment and quantitative assessment of flame status. Extracting flame geometric features requires an experimental environment and must be obtained in a specific experimental burner. It is worth mentioning that most flame image studies, including the above-mentioned articles, are based on experimental equipment in the laboratory to study the relationship between flame images and boiler combustion status. Few scholars have investigated the relationship between flame image studies on real industrial burners and the combustion status of industrial boilers. For example, Sun et al., 12 Liangyu et al., 13 and Tang et al. 14 focused on the industrial furnaces and used historical data collected from the distribution control system to establish corresponding prediction models. But they ignored the more relevant information about the flame images inside the furnace. The flame image information is not closely linked to the industrial boiler combustion process.
More theoretical studies on the flame combustion state can be performed in the laboratory. Experimental burners are simplified versions of industrial burners, which are difficult to combine with the operating parameters of industrial burners. Experimental burners control the flame combustion process by controlling the fuel quantity and ventilation rate. The flame combustion efficiency is judged by detecting the O 2 content of flue gas and flame radiation temperature. Industrial burners usually contain dozens of measurable state parameters. It is difficult to measure interference quantities, making the control process of industrial burners complex. Additionally, the camera position of an experimental burner is carefully designed to collect a complete flame image, whereas an industrial burner needs to be modified to obtain the flame image. The camera installation position is more constrained, and many flame features are difficult or even impossible to collect. Therefore, it is challenging to build a model that better fits the industrial combustion process for industrial boiler burner flame image acquisition and analysis.
To address the problems of control and flame feature acquisition in real industrial boilers, this study uses a large thermal power plant in Henan, China, DG2060/26.15-II2 type 660 MW ultrasupercritical boiler No. 2, as the basis for research. A 24 fps industrial camera was installed on the boiler wall to gain the flame image information. The frames of the flame images were sampled near the boiler operating parameter acquisition point (the operating parameters are the contents of O 2 and NO x ), Gabor filtering was performed in three RGB layers to enrich the texture features in each direction, and a GLCM was used for feature extraction. 15 PCA dimensionality reduction was performed after averaging all GLCM features within the video segment. The reduced-dimensional flame features were associated with the matching boiler operation parameters through the gated recurrent unit (GRU) network, the flame features are the input, the operating parameters are the output, and a regression model of the boiler combustion process was established. The mutual information on the dimensionality-reduced feature values and boiler operation data proved that the texture feature algorithm proposed in this paper could effectively characterize the physical significance of the flame combustion state. The combustion area of the furnace is the source of NO x generation and the concentration of chemical reactions in the furnace. Effective flame characteristics provide a certain degree of visualization of the current state of combustion in the furnace. Therefore, the combination of flame image feature extraction techniques and deep learning algorithms is of some significance for the study of boiler combustion monitoring and control. Section 2 introduces the selected 660 MW ultrasupercritical boiler, Section 3 introduces the flame image feature extraction algorithm based on a Gabor filter and GLCM, Section 4 is an experiment to prove the effectiveness of the algorithm using the mutual information on the obtained features with O 2 and NO x and the GRU neural network to associate the flame texture features with the boiler combustion data, and Section 5 is a summary.

DESCRIPTION OF THE BOILER COMBUSTION
SYSTEM In the combustion process of large industrial boilers, combustion efficiency is an especially important indicator, which represents simultaneously to the economy and the pollutant emission of the combustion process. The O 2 and the NO x content of the boiler flue gas can directly reflect the adequacy of the combustion reaction. It is considered an important indicator for detecting combustion efficiency and is involved in combustion efficiency control. In conventional large boilers, the O 2 content is measured using a flue gas analyzer installed at the tail of flue and is used to control the secondary airflow. The NO x content data is from selective catalytic reduction (SCR) reactor inlet measurements for SCR ammonia injection control. These measurement points are not in the initial area of the pulverized coal combustion reaction. The measurement results obtained in this way have a significant lag. The lag of the O 2 and the NO x content are 20 s and 8 min, respectively (the delayed data is from the industrial field). If these measurements are used directly, owing to the lags, the feedback control loop based on O 2 content tends to overcompensate, oscillate, or even exhibit unstable control quantities and make the SCR system ammonia injection delay. In contrast, the combustion zone of the furnace as the initial area of the reaction enables direct access to no-delay information. Thus, the flame image characteristics from it can reflect the flame combustion state and predict the components of tail flue gas without delay. It also shows the advantages of high real-time performance, both for combustion state monitoring and oxygen content cascade combustion control.
In this paper, a large thermal power plant DG2060/26.15-II2 type 660MW ultrasupercritical boiler No. 2 in Henan Province is studied, as shown in Figure 1.
The boiler uses butt-wall combustion technology with burners arranged in three layers on the front and rear walls and six on each side of each layer as well as six overfire air nozzles on each side to optimize the combustion process. The operating parameters of the boiler were measured every 30 s. To capture the image of the flame combustion, CCD cameras were installed above the first and fourth burners in each layer. The cameras were protected from high flame temperatures by an air shunt cooling device, and an antisaturation filter was installed in front of the lens. The camera acquired 24 frames per second with a resolution of 720 × 480 pixels. Here, the flame images are mainly from the fourth burner on the third layer of the front wall. A full top-down image of the flame at the lower two burner outlets can be captured, and gaining a portion of the combustion image in the center of the chamber is possible. The image at this location is typical. Figure 2 shows the original image captured by the camera and the image processing system. The camera captured the original image of the flame and disassembled the color according to red, green, and blue to retain the maximum color information on the flame. Subsequently, the image went through a texture feature extractor consisting of a Gabor filter and GLCM. A total of 12 Gabor filters (3 scales × 4 angles) were selected to highlight the texture features in each direction and for each scale. The boiler acquired operational parameters every 30 s, and the texture feature extractor was applied to each image frame, so that both the data volume and dimensionality yielded by the GLCM were massive. To reduce the load on the GRU network, a data compressor consisting of data averaging and data PCA was used for the data generated by the GLCM. The amount of data was significantly reduced while retaining 99.9% of the texture features of the image. Subsequently, the flame combustion regression model was obtained by associating the boiler operation parameters and flame texture features through the GRU network. It is worth mentioning that, as can be seen from the original flame images, the entire image acquisition system was based on modifying traditional industrial boilers. Therefore, limited by the possible camera installation locations, it was not possible to capture a single, complete image of the flame, which is an important reason for not using the geometric features of the flame.

FLAME IMAGE TEXTURE FEATURE EXTRACTION
BASED ON GABOR-GLCM Similar to the flame temperature detector and flue gas composition analyzer, a CCD camera is used to detect the flame combustion status. However, the camera captures a single frame of the flame image containing 720 × 480 pixels, with 3 color data points of each pixel and 24 frames per second. The total data is 24 883 200 per second, which is a wide variation from other sensors. A large number of variables cannot directly represent the flame-burning characteristics. Highlighting and extracting a certain feature of an image is the most effective way to identify the burning state of a flame.
Texture features are an important visual cue describing recurring local patterns and the arrangement rules of images. 16 Texture analysis techniques have been an important research topic in the fields of computer vision and image processing. 17−20 For example, Dhanasekar et al. 17 used texture analysis to describe and judge the mechanical surfaces. Brodic et al. 18 provided a novel method to identify language by using script texture analysis and computed the co-occurrence matrix to calculate the texture features. Tan et al. 21 introduced local ternary patterns, incorporating two complementary source-Gabor wavelets and local binary patterns (LBP), to identify face recognition and improve the face verification rate. Texture feature extraction has been widely used in different fields. In this study, we selected a texture feature extraction method for flame images based on a Gabor filter and GLCM, 15 which highlighted and scalarized the texture features of images, respectively. This method is illustrated in Figure 3.
3.1. Highlighting Texture Features of Flame Images Using Gabor Filter. In the image processing field, Gabor is a function that can be used to describe image texture information. 19 The frequency and direction of a Gabor filter are similar to those of the human visual system, which is particularly suitable for texture representation and discrimination. The Gabor filter is the result of the convolution of a Gaussian function and complex sine function in the Fourier domain. 22 The representation of the 2D Gabor filter in 3D space is shown in Figure 4.
The function expression of a 2D Gabor filter is as follows where the complex sine function is (x 0 , y 0 ) is the center point of the Gaussian kernel, θ is the rotation direction of the Gaussian kernel, (σ x , σ y ) is the scale of the Gaussian kernel in two directions, (u 0 , v 0 ) are the frequency domain coordinates, and K is the amplitude scale of the Gaussian kernel. The 2D Gabor kernel is used to convolve with the image to highlight the image texture features. The specific scale and orientation of the 2D Gabor kernel can capture the desired frequency response of the image along with this orientation. The overall frequency information on the image can be described using 2D Gabor kernels of multiple orientations and scales.
To preserve the texture features of the image as much as possible, a Gabor filter with four angles and three scales was selected in this study. Figure 5 shows the result of convolving a certain frame of the original image with the Gabor kernel. The angles of the Gabor filter are 0, 45, 90, and 135°and the scales are 13 × 13, 11 × 11, and 9 × 9 pixels. The result in Figure 5 is the convolution of the Gabor kernel at 0°for images R, G, and   B. It is worth mentioning that some interference information may occur due to the compression of flame video data in the industrial field and unstable hot air flow inside the boiler. Some clutter appears after convolving the image with different kernels. Because of the extent of variables in video images 5 s before and after the output point, 51 840 variables were used to predict that point in this study. Averaging and PCA processing for such a large amount of data can neutralize the disturbing effects of randomness.
3.2. Scalarization of Image Texture Features Using GLCM. A GLCM is often used to describe texture features, formed by the repeated alternation of grayscale on an image. 15 Therefore, there must be a certain grayscale relationship between two pixels separated by a certain distance in the image, and the study of these relationships is the concept of a grayscale co-occurrence matrix. 23 The essence of a GLCM is to count the frequency P(i, j, d, θ), of the simultaneous occurrence of image elements (x + D x , y + D y ) with distance d and grayscale j, starting from the image element with grayscale i(position x, y). 23,24 The mathematical expression is as follows x y x y (5) where x and y are the pixel coordinates of the image, and i and j are the gray levels.
To describe the texture more intuitively with the cooccurrence matrix, statistical parameters reflecting the matrix condition can be derived from it, and the following six are used in this study.
(c) Homogeneity where μ i and μ j are the means of the co-occurrence elements, and σ i and σ j are the standard deviations along the respective axes.
The above six texture statistics describe most of the texture relationships of the image in a certain direction and certain scale; in fact, similar to a Gabor filter, a GLCM also uses a total of 12 different directions and scales, as shown in Figure 5. Not all statistics can characterize the flame-burning features; the irrelevant variables are automatically filtered in the subsequent data compression process.
3.3. Data Compression of Flame Image Features. The texture feature extraction method for a single-flame combustion image frame is introduced above. However, the boiler studied in this paper obtained data every 30 s, which means that not every frame has boiler operation data corresponding to it. If only the images and their data at the data acquisition point are associated, the contingency of the flame will have a significant impact on the results. Because of the inertia of the disturbance occurring throughout the boiler combustion system, the flame combustion images do not change significantly near the data acquisition point. To reduce the chance and influence of the disturbance generated by the image acquisition process on the effective features, feature extraction was performed on the images near the data acquisition point for a period of 10 s.
As shown in Figure 6, such a large number of images will generate massive GLCM statistics, that is, 51 840 variables characterizing 1 point of boiler operation data. A simple analysis shows that 51 840 variables contains an enormous number of interfering and repetitive quantities, which would cause a huge computational load and high overfitting risk if fed into the GRU network. The weighted average method is an effective way to address disturbances.
where μ = 120 and σ = 50. The image corresponding to the location of the boiler data acquisition point (the 120th frame image) has the highest weight, and the images before and after this point enjoy decreasing weights on a sheet-by-sheet basis.
The Gaussian-weighted average method not only averages the 240 frames but also changes the percentage of contribution of different pictures to the average feature, which is more in line with common sense. However, even for a single image sample, it still contains 216 features each with a large amount of repetitive data. PCA is a multivariate statistical analysis method. The  15) where N = 1000 is the number of samples and m = 216 is the number of features of the flame. A central normalization process was performed to generate a standard matrix. (16) where X* is the normalized matrix, i = 1, 2, ..., N, j = 1, 2, ...m, and x j and s j are the mean and variance of x j , respectively. Establishing the correlation matrix R The eigenvalues λ 1 > λ 2 > λ 3 ...> λ m of the matrix R and the corresponding eigenvectors u 1 , u 2 , ..., u m were obtained. The cumulative variance contribution was calculated to determine the number of principal components.    (20) 4. REGRESSION MODEL BASED ON GRU AND EXPERIMENT RESULTS 4.1. Regression Modeling Based on Flame Characteristics and GRU. The process of disturbances occurring throughout the boiler combustion system has a certain degree of inertia, such as changes in pulverized coal density, load, air temperature, and humidity. Therefore, if the traditional backpropagation (BP) neural network is used, the time correlation between variables is ignored. In this study, the GRU network was selected as the regression algorithm for the image features and boiler sampling variables.
The internal structure of the GRU network is illustrated in Figure 7. Unlike the traditional BP neural network, the output y t of the GRU network is not only determined by x t but also related to h t−1 , which is a function of x t−1 . Therefor, the GRU network retains timing information. 25 The GRU network is similar to the long short-term memory (LSTM) network 26 in that it contains both a reset gate and an update gate, whereas the 1 − z part is equivalent to the forget gate in the LSTM. The training speed of the GRU network is much faster than that of the LSTM, which is very important for the training process of large samples. 27,28 As shown in Figure 7, a GRU cell has only two gates: a reset gate (z) and an update gate (R) without an independent storage unit. This is the network topology of the GRU structure. The updating formula for the model gating is as follows where r t and z t are the states of the update gate and reset gate at a time t, respectively, W Z and W r are the weight matrices of the update and reset gates, respectively, h t and ′ h t are the states of the hidden layer and candidate hidden layer network, respectively, • is the dot product, and σ()is a sigmoid function. According to eq 22, the output of the GRU network is shown in eq 23 In summary, for both the feature extraction and modeling, there are three main stages for the regression of the O 2 and NO x content based on the image of the boiler combustion.
Stage 1: Using different angles and sizes of Gabor filters highlight and enrich the image texture features near the output in the RGB channel. Then, six texture features statistics based on the Gabor are described by GLCM.
Stage 2: The disturbing effects of randomness for such a large amount data can be neutralized by the Gaussian-weighted average method. The PCA dimensionality reduction is used after the averaged features.
Stage 3: The extracted texture features of flame image are used to train the GRU network and construct the O 2 and NO x content regression model.
The entire training process can be applied directly. The trained regression network can directly output the O 2 and NO x content of the flue gas in the current combustion state. The boiler exhaust gas composition detection process contains the lag, 13 which has been eliminated by using the regression model in the study when associating the exhaust gas composition data with the flame image features. It helps offset the impact of the lags on the combustion monitoring and control system.

Data Collection and Network Configuration.
To determine the flame-burning status based on the flame texture features, a large amount of measured data from the boiler is required as the target of the regression algorithm. However, the distribution of the measured data and the validity of the flame image texture features both have an impact on the regression algorithm. If the measured training data are too average and regular, it will be difficult to generalize the model obtained by the regression algorithm to more realistic working conditions. If there is no correlation between the image features and the measured data, it will make the regression algorithm overfit from the beginning and lose the meaning of regression. The measured O 2 and NO x data from a total of 1000 collection points for 8 h of boiler operation, as shown in Figure 8, were selected as the regression targets. Among them, the fluctuation amplitude of O 2 and NO x content reached 132 and 51%, respectively, and there was no obvious correlation between the two. This data distribution covers most of the operating conditions of the boiler, which makes the subsequent experimental work meaningful.
The collected data from 8 h of boiler operation were divided into two parts: a 6.6 h training set and a 1.6 h test set. The data in the test set were independent and did not participate in any training process. As the GRU model is a time-series model, the latter 200 continuous sampling points were chosen as the test  (23) whereŷ i is the predicted value, y i is the true value, and n is the number of samples. The training process with O 2 and NO x content as regression targets is shown in Figure 9.
The To avoid overfitting, the loss rate of the GRU model hidden layer was set to 0.2, and the initial learning rate was set to 0.05. To train the model, two targets were predicted, training the data for 10 000 epochs, as shown in Figure 9 to find the optimum epochs.
The regression between image features and O 2 content started to overfit at approximately 1000 training epochs, with a MAPE min of 7.5%. The regression between image features and NO x content starts to overfit at approximately 7600 training epochs, with a MAPE min of 10.2%. Thus, the O 2 content prediction model was the output model obtained at 1000 epochs, whereas the NO x content prediction model was obtained at 7600 epochs.   parameters and the texture features extracted from the flame images. Correlation analysis is an important tool to prove the validity of the features extracted by the algorithm in this paper. It is also necessary before conducting the neural network regression algorithm. For two discrete random variables X and Y, let p(x, y) be their joint probability distribution function of X and Y and p(x) and p(y) be their respective marginal probability distribution functions. The mutual information between them can be expressed as i k j j j j j y The O 2 and NO x content data of 1000 acquisition points were used to obtain mutual information with the corresponding eight image texture feature parameters. The results are shown in Figure 10 where the horizontal axis is the 8 feature values, arranged from highest to lowest in terms of variance contribution, and the vertical axis is the information correlation. It can be seen that features 1, 3, and 4 contain a high correlation with oxygen content data. Features 2 and 3 are highly correlated with NO x data. Thus, verifying the flame feature extraction algorithm proposed in this paper.

Results of the O 2 and NO x Content Regression Model.
To verify the advantages of the proposed algorithm, a comparison test was performed between the feature detection algorithm proposed in this paper and other algorithms, which are PCA processing directly to the image and PCA processing after applying a GLCM directly to the image. Figures 11 and  12 show the GRU network results of O 2 and NO x content prediction trained with different image feature inputs by three feature extraction. A prediction performance of the GRU network using the flame features extracted by Gabor-GLCM is better than other extracted algorithms. Especially, the prediction results of the O 2 content are better than the NO x content clearly shown in Figures 11 and 12. The GRU network based on PCA and PCA-GLCM feature extraction methods cannot achieve excellent prediction for the contents mainly because the features are not suitably extracted. Moreover, the online prediction results of these three methods are compared and listed in Table 1. The evaluation indices in Table 1 show that the extracted method of Gabor-GLCM is more suitable for the prediction of parameters during flame combustion than the other two methods. The MAPEs of the three algorithms for oxygen content prediction are 19.8, 14.8, and 7.5%, and the MAPEs of the three algorithms for NO x content prediction were 17.4, 13.9, and 10.2%, respectively. Therefore, the GRU network based on the features extracted of the Gabor-GLCM exhibit more accurate prediction results than the other traditional methods in terms of the prediction of the O 2 and NO x contents in the industrial boiler.

CONCLUSION
In this paper, we proposed a Gabor filter and a GLCM-based flame image texture feature extraction method in an industrial boiler. This method enhanced the texture of the flame and transformed it into scalarized statistical values. The statistical values were compressed into eight dimensions by Gaussian-  weighted mean and PCA methods, which not only sped up the training of the neural network but also maintained 99.9% of the flame image texture features. Finally, the predictions for O 2 and NO x content reached deviation values of 7.5 and 10.2%, respectively, on the test set by the GRU network. Further, it was verified by comparison experiments that the flame image feature detection algorithm proposed in this paper was significantly better than the method of direct PCA on the image and the method of PCA after performing GLCM statistics on the image.
Existing flame image detection algorithms are mainly developed using experimental equipment, but actual boilers are different from experimental equipment, and flame image acquisition is limited by the camera installation locations. In this paper, we analyzed and experimented on a DG2060/ 26.15-II2 type 660 MW ultrasupercritical boiler and proposed a texture-based feature detection algorithm under the premise that it is difficult to collect complete flame geometric features. In the meantime, the images mainly originate from a single burner outlet, although it is possible to characterize the current combustion state to a certain extent. And it is still necessary to explore the influence of multiple-burner outlet image features on the state parameters. This will be the main direction of future research. The image feature extraction method proposed in this study can, to a certain extent, effectively help other scholars to visualize and analyze the level of chemical reaction of pulverized coal combustion. At the same time, it can also provide a new solution idea for monitoring chemical reactions in the furnace. The feature detection algorithm proposed in this paper still has some generalization significance for other industrial boilers. The optimization of the image acquisition process and boiler combustion control combined with images are future research directions for the authors of this paper.