{'fraction_non_alphanumeric': 0.03143765453903214, 'fraction_numerical': 0.012998940303779584, 'mean_word_length': 4.628628230616302, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Automatic State Matching Gaussian Process Ensemble for Wood Planer Control

: Wood planers are high speed sophisticated lumber ﬁnishing machines that are diﬃcult to operate and for which the available data shows complex, non-linear patterns. We present a machine learning approach to build a control loop for an industrial wood planer. In order to predict the thickness of the outgoing boards with better accuracy than the industry standard whilst allowing dynamic planer adjustments, we use an ensemble of Gaussian Processes with a specialized weighting scheme we call Automatic State Matching . It reduces the prediction error by 39% compared to current industrial practice.


INTRODUCTION
Canadian forest-product companies face serious challenges due to current trends towards globalization and worker shortages. Experienced planer operators are increasingly difficult to find. Although planers are highly flexible machines, adjustments are mostly performed only when a problem is detected and the planer is idle. This leads to the production of boards with less desirable dimensions. The infrequency of adjustments is mostly due to the difficulty of finding experienced operators, and the high cost of mistakes. More frequent and dynamic adjustments could keep the outgoing boards closer to the desired dimensions which would generate additional profits, and automated adjustments could alleviate the issue of the lack of workforce. Following this idea, we address the following research question: Would it be possible to design a machine learning based control loop in order to automate a high-speed sophisticated wood planer?
As an answer to that question, we propose a novel algorithm which will be used in a control loop to adjust the planer in real time. Historically, the main tool used in industrial control loops is PID control (Johnson and Moradi, 2005). In recent years, trends towards data collection have facilitated the rise of predictive machine learning models in order to provide proactive control (Camacho and Alba, 2013) and process control in production planning (Usuga Cadavid et al., 2020). Other related approaches are reinforcement learning (Barto, 1994) where performance is improved through trial-and-error, and deep reinforcement learning which dramatically extends the level of complexity in process dynamics that can be learned compared to traditional reinforcement learning (Lillicrap et al., 2015). Our partners from the industry reported that the dynamics of the wood planer are too complex for optimal PID control. In addition, due to the high financial costs of mistakes (and the absence of a realistic simulator capturing the high variability/instability of the process), we cannot experiment directly on the planer which would be needed for reinforcement learning (deep or otherwise). Thus, we focus on model predictive control. We propose to model the system as an ensemble of Gaussian Processes (one for each previous production batch). Weights are dynamically computed to match in real time the current batch to the previous batches. This allows dealing with the high variability of the process.
The paper is organized as follows: In Section 2, we review the necessary concepts and challenges related to the task of building a control loop for a wood planer, as well as the available industrial data. Section 3 details the proposed approach. In Section 4, we explain the experiments we performed to evaluate the proposed approach. The results of those experiments are discussed in Section 5. We conclude in Section 6.

PRELIMINARY CONCEPTS AND CHALLENGES
A lumber planer is used to trim rough boards (lumbers) to the desired thickness and/or width in order to be sold on the market. The value/quality of a lumber depends on  Figure 1 shows a planer diagram with the relevant components along with the number of data points (features) we consider for each component. There are 50 control settings which can be adjusted by the operators (e.g., the position of the knives). The built-in sensors of the planer provide 88 data points during the processing of a board (e.g., the temperature inside the planer). External sensors provide 10 more informative measures about the board before it is processed and 7 afterwards (e.g., the temperature of the board before and after being processed).
When the machine is idle for more than 60 seconds, it is highly likely that a change has occurred (mechanical or parameters adjustments by the operators or, changes in the type of wood that is being processed). Consequently, sequential data acquired without any interruptions longer than 60 seconds is defined as a batch. Figure 2 shows the thicknesses of boards for two consecutive batches where the adjustable settings are the same. We can see that boards are, on average, thicker in the second batch, despite no change in adjustable settings being recorded in the database.
An important adjustable/controllable setting of the planer is called the "OFFSET". It can be used to move the knives closer or further to the wood in order to adjust the expected thickness. However, this is infrequently done as there is a lot of noise in the data and operators are not confident enough to do it in real time, neither the mills are confident enough to implement a traditional feedback loop for the same reason.

PROPOSED APPROACH: SMART PLANER CONTROL WITH GAUSSIAN PROCESSES
Our data was collected in a sawmill in Quebec, Canada. We focused on a single product (one of the most available standard-size lumber products of North America called "2×4×96") for which we acquired a total of 155 data points each time a lumber is processed (148 before the board is processed and 7 afterwards). Preliminary analysis showed that the distribution of the boards dimensions coming out of the planer can be dramatically different from Thickness of "2×4×96" boards for two consecutive production batches with identical adjustable settings. Outliers were cut out to make the figure more readable.
one batch to another. As an example, Figure 2 reports board thickness for two consecutive batches. Although no settings were changed in the wood planer between those batches, we can yet observe two very different processes. The non-stationary nature of the data could be due to the fact that some important variables are unknown, e.g., the type of wood, the humidity in the wood, the wear and tear of the planer knives and changes in mechanical parameters. This leads us to believe that using a simple machine learning model trained on all available data without taking into account its non-stationary nature (and the concept of batches) could be difficult.
Therefore, we propose an approach based on Gaussian Processes (GPs). Gaussian processes have found a wide range of applications in systems control (Kocijan, 2016). Due to the amount of data, the non-stationary nature of We use the history of the n − 1 previous boards from the current batch in order to compute the likelihood L 1,n , L 2,n , . . . , L M,n of each Gaussian Process GP 1 , GP 2 , . . . , GP M . We also use the features from the planer associated with the n th board in order to compute a predictionŷ * 1,n ,ŷ * 2,n , . . . ,ŷ * M,n from each Gaussian Process. We then compute weights w 1,n , w 2,n , . . . , w M,n for each prediction according to the likelihood of the associated Gaussian Process in order to compute the final prediction of the thicknessŷ * n .
the process and performance constraints, using a single GP would likely prove inefficient in practice (the poor predictive performance of that approach is also highlighted by our results). However, using an ensemble of GPs, trained on partitions of a larger dataset (i.e., the batches), allows for a non-negligible reduction in computational complexity and gains in scalability versus using a single GP trained on the whole dataset (Tresp, 2000). Consequently, we train a different GP for each batch of the training dataset.
For a new batch, we feed the features of the current planer/board into each model and each one produced a prediction. After seeing the actual thickness of the board, the weight associated with each GP is dynamically adjusted to represent how well the GPs matches the batch. Similar weighting schemes have recently produced interesting results in many control applications, e.g., (Nguyen-Tuong et al., 2009;Schneider and Ertel, 2010;Gramacy and Apley, 2015).
In our case, we dynamically adjust the weights of the ensemble each time a new board is seen. Previous work has been done on dynamic weights, especially to model non-stationary data (Kolter and Maloof, 2007;Yin et al., 2015;Li et al., 2020). In our case, the adjustment is made probabilistically by weighting each GP proportionally to its likelihood (how well it fits the boards previously seen in the current batch). Thus it probabilistically "matches" the new batch to a convex combination of our models trained on past batches. We named this weighting scheme Automatic State Matching (ASM). The procedure is illustrated in Figure 3 and Figure 4.
More formally, as our prediction for y * n , the thickness of the n th board, we use the mean of the predictive distribution of the ensemble, which is obtained by using the weighted individual model outputs induced by x * n , the vector representing the current features of the planer. The prediction produced by our ensemble follows a univariate Gaussian distribution with mean: where X i , Y i and K i are respectively a vector of the features used as input during the training of the model m i , the actual thicknesses of the associated outgoing boards and the kernel function of the model m i as described in (Rasmussen, 2004). The weights for each model m i for the n th board, w n,mi , are computed after each board as follows: where L is the likelihood function (Rasmussen, 2004) and H n is the history of the batch (the n − 1 pairs of input vectors and thicknesses (x * j , y * j ) where j = 1, . . . , n − 1).

EXPERIMENTS
The data for 118,792 boards (83 batches) was gathered during a 13-day operation period. We split our dataset into a training set consisting of the data from the first 10 days and a testing set containing the data for the last 3 days. As a result, we have the data for 97,512 boards in our training set and 21,280 boards in our testing set. Our training set contains 73 training batches and our test set contains 10 batches.
Many of the features are redundant. Experts identified 10 adjustable settings that are key in optimizing thickness. We also identified 8 key informative features through data exploration and by discussing with the manufacturer. These 18 (10 + 8) features are the input of our model and the most reliable sensor for thickness will be our target variable. Table 1 summarizes the features.
In order to obtain our ensemble, a Gaussian Process is fitted on each training batch. It uses an Automatic Relevance Determination (ARD) Matérn kernel in order to identify which features are relevant to the batch being trained (MacKay et al., 1998). ARD kernels have been successfully used in contexts such as predicting lithium-ion batteries aging (Liu et al., 2019), soft-sensor modeling (Huazhong, 2007), and optimization of timedependent fermentation control strategies (Coleman and Block, 2006).
For each batch in the testing set, we evaluate the mean absolute error (MAE) of our predictions w.r.t. the raw output thickness data. This metric is targeted in practice by domain experts hence our choice.
As recommended by experts from the industry, we will use as a baseline the practice of simply supposing that a board will have the same thickness as the previous board (hereafter "Last Board"). Based on their recommendation, we will also evaluate the performance of a moving average model with a window of size 5.
As a comparison, we report the results obtained by using an equal weight for each model (GPE) and the results obtained by using the sole model for which the ASM weighting scheme assigns the largest weight (Best GP). Finally, we tried many models from the machine learning literature trained on the whole training dataset, notably many variations of linear regression, support vector machines, decision trees and neural networks. Among these "classical" models, we present only the results of XGBoost since it provided the best results. Training a single model (XGBoost) or using an equal weight in an ensemble of Gaussian Processes (ASM-GPE) also produces an increase of the MAE (respectively 20.27% and 59.28%). Due to the high variability observed between the batches, this was expected.

RESULTS
Selecting the single "best" model from the GPs ensemble (Best GP) allows a reduction in the MAE of 27.25% on average. However, for Batch 4 the MAE is still lower for the Last Board model.
Finally, our approach (ASM-GPE) produced the best results with the lowest MAE for every batch and an average reduction of the MAE of 39.63%.

CONCLUSION
ASM-GPE allows for some unique advantages in our context, namely we explicitly know which past batches collectively best matches the current one. In practice, this valuable information can be used in various ways, e.g., raising flags when the weight associated with a GP trained on a past batch known to have preceded mechanical issues exceeds a certain threshold. In addition, our approach produced the best results with the lowest MAE for every batch, an average reduction of the MAE of 39.63% w.r.t. the current standard using the Last Board thickness as a predictor. As a result, the approach is currently being implemented in an actual mill to suggest "OFFSET" adjustments to the operators. After this first implementation, the full automated control loop will be implemented.
Finally, although the approach is currently being used to adjust a single variable, i.e., the offset, it is designed to be able to assist in the optimization of all settings by extracting information as to which adjustable settings have the greatest impact in the current state. We can extract this information from the most likely GPs' ARD Kernel and then use the model itself alongside classical