{'fraction_non_alphanumeric': 0.04412854068926594, 'fraction_numerical': 0.012628311404282338, 'mean_word_length': 4.4984484096198605, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Data-driven design of explicit predictive controllers using model-based priors

In this paper, we propose a data-driven approach to derive explicit predictive control laws, without requiring any intermediate identification step. The keystone of the presented strategy is the exploitation of available priors on the control law, coming from model-based analysis. Specifically, by leveraging on the knowledge that the optimal predictive controller is expressed as a piecewise affine (PWA) law, we directly optimize the parameters of such an analytical controller from data, instead of running an on-line optimization problem. As the proposed method allows us to automatically retrieve also a model of the closed-loop system, we show that we can apply model-based techniques to perform a stability check prior to the controller deployment. The effectiveness of the proposed strategy is assessed on two benchmark simulation examples, through which we also discuss the use of regularization and its combination with averaging techniques to handle the presence of noise.


Introduction
For its capability of handling constraints, Model predictive control (MPC) is a widely employed technique for advanced control applications (see, e.g., [14,19,21,23,24]). Due to the increasing complexity of the systems to be controlled, the model required by MPC is often no longer parameterized from the physics but learned from data in a black-box fashion. However, this identification step generally takes the lion's share of the time and effort required for control design. In the last years, research has thus benched into two main direction. On the one side, efforts has been focused on improving and easing learning procedures [9,20]. On the other side, many approaches have been proposed to directly employ data for the design of predictive controllers, while bypassing any model identification step. Among existing works in this direction, we recall the foundational contributions of [5,10], which have been extended to handle tracking problems [4], to deal with nonlinear sys- tems [6], and to improve the performance in the presence of noise [11,13], just to mention a few. For both traditional and data-driven predictive controls, the computational effort required to solve the related constrained optimization problem is known to be a potential limit for its application, especially for fast-sampling systems. Nonetheless, when the MPC problem is relatively small, i.e., one has to control a low order system and/or the prediction horizon is relatively short, this limitation can be overcome by explicitly deriving its solution [3]. In fact, when the cost of the optimization problem is quadratic and the constraints are linear, the explicit solution of MPC is known to be a Piece-Wise Affine (PWA) state feedback law.
In this work, we propose an approach to directly learn explicit predictive control laws from data. More specifically, we initially build on the foundational results in [12,26] and on model-based priors (namely, the aforementioned fact that the optimal solution of linear MPC is PWA [3]), to construct a data-driven, multi-step closed-loop predictor. The latter is exploited to construct a fully databased optimization problem, yielding the estimation of the parameters of the optimal predictive control law. As a by-product, we obtain a data-driven description of the closed-loop behavior, which can be used in combination with existing model-based techniques (see, e.g., [22]) to check the stability of the system controlled with the data-driven explicit controller, prior to its deployment.
As far as we are aware, this is the first time a data-driven predictive control strategy is provided together with a preliminary assessment of its closed-loop performance.
We should mention here that an early attempt to obtain a data-driven counterpart of explicit MPC was already carried out in [25]. However, the strategy proposed therein relies on an implicit open-loop identification step. Another method was presented in [8], by relying on the behavioral predictor used in [5,11]. Even though that method involves no open-loop identification phase and it does not require the state to be fully measurable, the latter does not leverage on priors coming from model-based analysis and design. Nonetheless, by leveraging on priors, we are here able to retrieve a data-based characterization of the closed-loop and, thus, practically assess its stability in a data-driven fashion prior to its deployment. This is instead not possible in [8], where stability cannot be directly assessed nor it is theoretically guaranteed in presence of noisy data.
The remainder of the paper is organized as follows. The targeted problem is formalized in Section 2, while all the steps required to obtain its data-driven formulation from priors are introduced in Section 3. The explicit control law is derived in Section 4, where we additional discuss practical aspects for its implementation, certified deployment and noise handling. The effectiveness of the proposed strategy is assessed on two benchmark examples in Section 5. Section 6 concludes the work and indicates some directions for future research.
Notation We denote with N 0 the set of natural numbers, that includes zero. Let R, R n and R n×m be the set of real numbers, column vectors of dimension n and n × m dimensional real matrices, respectively. Given B ∈ R m×n , its transpose is B ⊤ , its Moore-Penrose inverse is B † and, when m = n its inverse is indicated as B −1 . Given a vector v ∈ R n , [v] i:j indicates its rows from i to j, with i ≤ j ≤ n. For a matrix B ∈ R n×m , [B] 1:i,1:j denotes a sub-matrix comprising the first i rows and j columns of B, with i ≤ n and j ≤ m. Identity matrices are denotes as I, while zero matrices and vectors will be denoted as 0. If a matrix Q ∈ R n×n is positive definite (positive semi-definite), this is denoted as Q ≻ 0 (Q 0). Given a vector x ∈ R n , the quadratic form x ′ Qx is compactly indicated as x 2 Q . Given a signal {ν t ∈ R m } t∈N0 and 0 < L < T , we denote with N 0,L,T −1 ∈ R mL×T −L+1 the associated Hankel matrix while, for i, j ∈ N 0 , we introduce Consider the class S of discrete-time linear, time invariant (LTI), controllable systems with fully measurable state, i.e., S : where x(t) ∈ R n denotes the state of S at time t ∈ N 0 , u(t) ∈ R m is an exogenous input and y o (t) ∈ R n is the associated noiseless output. Let us consider the following model predictive control (MPC) problem: The objective of (4) is to steer both the (predicted) statẽ x(k) and the inputũ(k) to zero, over a prediction horizon of prefixed length L > 0. Optimality is indeed dictated by: (i) the distance of the predicted state from zero, penalized with Q 0 over the whole horizon except for the terminal state (weighted via P 0), and (ii) the control effort, penalized via R ≻ 0. Meanwhile, a set of n c polyhedral constraints dictated by (4c) has to be satisfied, with C x ∈ R nc×n , C u ∈ R nc×m and d ∈ R nc , while relying on the latest information on the system (see (4d)). Assume also that the matrices A ∈ R n×n and B ∈ R n×m characterizing the dynamics of S are unknown, and that we can access a set of input/output data pairs D T = {U T , Y T }, where U T and Y T denote the available input and output sequences, respectively, satisfying the following assumptions.

Assumption 1 (Persistently exciting inputs)
The input sequence U T = {u(t)} T t=0 is persistently exciting of order n + L.
Assumption 2 (Noisy outputs) The output sequence Y T = {y(t)} T t=0 is corrupted by noise, namely where v(t) is the realization of a zero mean white noise with covariance Ω ∈ R n×n .

Assumption 3 (Sufficiently long dataset)
The length T of the dataset D T satisfies the following: The goal of this work is to directly exploit the available data to find an explicit solution to (4), under Assumptions 1-3, while bypassing any identification step.

Exploiting priors for explicit DDPC
To attain our goal, it is fundamental to replace the model in (4b) with an expression that directly depends on the available data, by also not forgetting what we have learned from model-based predictive control. To start with, we thus recall the following lemma, explicitly stating the form of the optimal explicit predictive controller in our setting [3].
Lemma 1 (On the solution of (4)) Let U denote the vector stacking the inputs over the prediction horizon, i.e., The optimal control sequence U ⋆ = U ⋆ (x(t)) solving (4) is a continuous piecewise affine (PWA) function of x(t).
Then, we generalize the one-step ahead predictor introduced in [12] to the multi-step case. To this end, it is important to recall that, thanks to Assumptions 1, 3 and the Fundamental Lemma [26], the following rank condition holds: We can now derive the multi-step data-based predictor as follows.
Theorem 1 (Data-based multi-step predictor) Let Assumptions 1 and 3 hold. Given x(t), the sequence of predicted statesX =X(x(t)) admits the following data-based representation: Proof: The proof follows the steps of the one in [12,Appendix B]. Specifically, let v ∈ R n+mL and S ∈ R n+mL×T be respectively defined as: with S being full row rank, i.e., rank(S) = n + mL. By the Rouché-Capelli theorem, for any given v, the equality v = Sα, admits infinite solutions α of the form with Π ⊥ S = (I − S † S) being the orthogonal projector onto the kernel of S. Meanwhile, based on the model in (4b), the predicted state sequence can be defined as a function of x(t) and U as: In turn, such a sequence can be recast as a function of α, i.e.,X = ξ Γ Sα = X 1,L,T −L α.
where the second equality straightforwardly follows from (10) and the definition of S. By replacing α with (9), we then obtaiñ This preliminary result allows us to exploit priors on the solution of (4) for the definition of the DDPC problem. In fact, according to Lemma 1, we can parameterize the control sequence U as: where K i ∈ R mL×n and f i ∈ R mL are the (unknown) feedback and affine gains characterizing the control law, dictates the associated polyhedral partition, for i = 1, . . . , M , and the amounts of modes M is dictated by the number of possible combinations of active constraints. Therefore, for a given state x(t), the input sequence is the affine function with K = K s(x(t)) and f = f s(x(t)) denoting the gains associated to the active control law, and with Based on this parameterization, we can compute a databased closed-loop characterization of the predictor in (4b), as outlined in the following theorem.
Theorem 2 (Closed-loop multi-step predictor) Let Assumptions 1 and 3 hold. Given x(t), the sequence of predicted statesX =X(x(t)) can be equivalently expressed as:X with G K ∈ R T −L×n and G f ∈ R T −L satisfying: where K and f characterize the local control law (12). Accordingly, the input sequence is given by: Proof: For the Rouché-Capelli theorem, there exists an T × n matrix G k and a T -dimensional vector G f such that (13b)-(13c) hold. Meanwhile, replacing (12) into the open-loop predictor in (8), the predicted state sequence can be represented as: By combining these two results, the closed-loop representation in (13) and the equivalent definition of the input sequence in (14) straightforwardly follow.
By leveraging on (13)- (14), we can now equivalently recast the predictive control task in (4) as an optimization problem with the closed-loop matrices G K , G f being the decision variables, as follows 1 : In (15), and Note that, the last constraints (see (15e)-(15f)) are introduced for the problem to be consistent with the closedloop representation in (13).
This shift from an open-loop predictor to its closed-loop counterpart allows us to directly learn the control law from data, and avoid any system identification step. (8) and (13) exactly hold in a noiseless setting only. As such, problem (15) is equivalent to (4) only when the available batch of data is noise-free.

Learning Explicit DDPC
To derive its explicit solution, the problem in (15) is manipulated to obtain a multi-parametric Quadratic Program (mp-QP). As a preliminary step, we condense the unknowns of (15) into a single variable: Accordingly, we can recast (15) as the following mp-QP minimize where and By focusing on H d in (17a), it can be proven that this weighting matrix satisfies the following lemma. Proof: This is a direct consequence of Assumptions 1 and 3, for which V ⊤ ∈ R 2(T −L)×mL is not full row rank.
As the cost should be strictly convex for a unique explicit solution to be retrieved, this feature of H d prevents us from deriving the explicit law. To overcome this limitation, we introduce a regularization term in the cost of (17), thus replacing the weight H d with: where γ > 0 is an hyper-parameter to be tuned 2 . The data-driven control problem then corresponds to the regularized mp-QP: .
(20c) 2 The cost has been normalized to ease the subsequent derivations.

Derivation of the explicit DDPC law
The introduction of the regularizer in (20) allows us to derive the explicit DDPC law through the manipulation of the Karush-Kuhn-Tucker (KKT) conditions associated with the new DDPC problem. To ease the computations, let us consider the following further assumption.
Based on this assumption, we now follow the same steps used to derive the explicit model-based predictive control law in [3].
The KKT conditions for the regularized DDPC problem in (20) are: where λ and µ are the Lagrange multipliers associated with inequality and equality constraints in (20b) and (20c), respectively. Let us focus on the i-th set of active constraints only, distinguishing between the Lagrange multipliers associated with a given active and inactive inequality constraints. We respectively denote them as λ i andλ i . It is straightforward to notice that the combination of (21b) and (21c) leads to the following condition onλ i :λ i = 0. By merging (21b) and (21e) for the i-th set of active constraints, it is also straightforward to show that the optimal solution G i (t) satisfies where andΞ d,i ,Ψ i andD i are the rows of Ξ d , Ψ and D coupled with the considered set active constraints. By leveraging on (21a), we can now express our optimization variable G i (t) as a function of the Lagrange multipliers, i.e., We can now replace the latter into (22) to obtain an explicit expression of the Lagrange multipliers as functions of the matrices characterizing (20): In turn, this allows us to explicitly retrieve G i (t) as: and the associated optimal input sequence as Thus, the input to be fed to the system when the i-th set of constraints is active is defined as: Through (21c) and (21d), we can finally define the polyhedral region associated with the considered combination of active constraints, which is dictated by the following inequalities: The complete data-driven expression for (11) is then straightforwardly obtained by following the above steps for all possible combinations of the active constraints. This operation ultimately yields an optimal input sequence U (x(t)) of the form: where M is given by the number of possible combinations of active constraints, U i corresponds to (26), can be easily obtained from (28). Consequently, the input to be fed to S starting from x(t) can be retrieved by evaluating the PWA law with u i (x(t)) given by (27) for all possible combinations of active constraints. 7. Merge polyhedral regions as in [2]. 8. Extract the first component of the optimal input sequence U (x(t)).

Remark 2 (On Assumption 4)
Although introduced to ease computations, we remark that Assumption ?? is not restrictive. Indeed, degenerate cases can be straightforwardly handled via existing approaches, e.g., see [3].

Remark 3 (Data-driven and model-based)
Within a noiseless setting, the results in Theorem 2 and the one-to-one correspondence between the chosen parameterization of the control law in (11) and its model-based counterpart guarantee the equivalence between (4) and (15). Therefore, when there is no noise, the data-driven explicit controller coincides with the E-MPC law as γ → 0.

Implementing Explicit DDPC
Based on the available batch of data and the features of the considered predictive control problem, the explicit DDPC law can be completely retrieved offline from the available measurements, as summarized in Algorithm 1.
Given the data, one has to initially construct the Hankel matrices needed to build the DDPC problem (see steps 1-2). Once all the possible combinations of active constraints have been detected at step 3 and degenerate scenarios have been handled (see step 5), at step 6 the local controllers and the associated polyhedral regions are retrieved according to (26)-(28). Lastly, at step 7, the optimal control sequence is simplified, by merging polyhedral regions whenever possible. After this step, the explicit optimal input can simply be retrieved by extracting the first element of the input sequence U (x(t)) (see step 8).
Once the explicit DDPC law has been retrieved, the computation of the optimal input at each time instant simply consists of a function evaluation. Specifically, one has to (i) search for the polyhedral region the current state x(t) belongs to, and (ii) apply the corresponding parametric law. We stress that this computational advantage is retained for simple control problems only (i.e., for short prediction horizon and small systems). Indeed, the complexity of the PWA law is known to rapidly increase [7] with the one of the DDPC problem to be solved, analogously to the model-based case.

Explicit data-driven predictive control and closedloop stability
When designing a controller in a data-driven setting, it is crucial to check the stability of the resulting closed-loop system before the controller deployment. Towards this objective, we now show how the peculiar features of the explicit data-driven predictive control can be leveraged in combination with existing techniques to devise an offline, data-driven stability test. To this end, let us assume that the i-th set of constraints is active and consider the following multi-step ahead closed-loop model: obtained by combining (13) with the result of our explicit derivation in (25), whereX stacks the state predicted by the learned closed-loop model, i.e., . . .
From (31), we can then isolate the learned one-step ahead closed-loop model, namelŷ When performed for each mode i ∈ {1, . . . , M }, these manipulations allow us to retrieve the datadriven closed-loop transition matrix A cl d,i for each i ∈ {1, . . . , M }. Retrieving these matrices ultimately enables us to apply model-based techniques, e.g., the ones presented in [22], to shed a light on the features of the final closed-loop. As an example, one can search for a matrix P ∈ R n×n satisfying the following sufficient conditions for asymptotic stability: or, alternatively, look for the set of matrices {P i ∈ R n×n } M i=1 verifying the following LMIs: which are also sufficient conditions for asymptotic stability.
Remark 4 (On the tuning of P in (4a)) When S is known to be open-loop stable, the terminal weight P 0 in (4a) is generally selected as the solution of the Lyapunov equation This equation can be directly translated into its datadriven counterpart by exploiting [12] as follows: with thus providing a data-based approach for the selection of this parameter.

Remark 5 (Hyper-parameter tuning)
The possibility of performing an off-line data-based stability check on the data-driven explicit law can be useful to preliminarily assess the effects of different choices of the tuning parameters Q, R and P in (4a) and γ in (19), allowing one to discard the ones resulting in a failure of the data-driven stability tests.

Regularization and noise handling
As highlighted in Remark 1, all the equivalences we rely on to derive the explicit predictive control law are verified when D T is noiseless. However, in practice, Ω in Assumption 2 is generally a non-zero matrix.
To cope with noisy data, we follow the footsteps of [5,13] and propose to leverage on the regularization term introduced in (19). Indeed, as in standard ridge regression [18], this additional element of the cost steers all the components of G(t) towards small values. In turn, this potentially limits the impact of noise on the constraints in (20b)-(20c) and, thus, on the final explicit law. This shrinkage effect is modulated by the regularization parameter γ, with the reduction in the magnitude of G(t) being stronger whenever large values of γ are considered. At the same time, γ implicitly changes the balance between the penalties in the original datadriven control problem in (15), with excessively high values of γ potentially driving the explicit data-driven controller far away from its model-based counterpart. The choice of this hyper-parameters thus becomes an important tuning-knob of the approach, requiring one to tradeoff between handling noise and keeping explicit DDPC as close as possible to the implicit DDPC problem.
Although the stability checks in (33)-(34) can be used to have a preliminary assessment on the effect of different choices of γ, at the moment this balance can only be attained through closed-loop trials for several values of γ. Such a procedure allows one to ultimately select the hyper-parameter that best fits one's needs, at the price of requiring closed-loop experiments that can be rather safety-critical in practice, especially when a simulator of the system is not available.
Whenever multiple experiments can be performed by feeding the plant with the same input sequence U T , the burden associated to the choice of γ can be alleviated by exploiting the features of Assumption 2 itself. In this scenario, one can indeed replace D T with the averaged with y (i) t denoting the output of the i-th experiment. Since the noise is assumed to be zero mean, the law of large numbers asymptotically yields As such, when the number N of experiments increases, the role of γ in handling noise is progressively less dominant. In this case, γ should then be used only to make the DDPC problem well-defined. Any small γ > 0 is acceptable for this purpose.

Numerical examples
The where x i (t) and w i (t) denote the i-th components of the state and the measurement noise, respectively. All computations are carried out on an Intel Core i7-7700HQ processor, running MATLAB 2019b.

Open-loop stable benchmark system
Let us consider the benchmark system described by: S (39) Our goal is to regulate both components of the state to zero, while enforcing the following box-constraint on the input: (40) Towards this goal, we collect a set D T of T = 100 input/state pairs, by feeding S with an input sequence uniformly distributed within the interval [−5, 5]. According to Assumption 2, the measured states are corrupted by an additive zero-mean white noise sequence, with variance yielding SNR = 20 dB. The parameters characterizing the DDPC problem to be solved are selected as in [3], namely L = 2, Q = I, R = 0.01 and P is chosen as the solution of the data-driven Lyapunov in (35). By setting γ = 10, the partition associated with the explicit data-driven predictive controller 3 is the one shown in Figure 1, which approximately correspond to that reported in [3]  This indicates that the explicit law preserves the stability of the open-loop system. Figure 2 report the trajectories of the state and the optimal input obtained over a noise-free closed-loop tests with the explicit data-driven law, which confirm its effectiveness and validate the result of the data-driven stability check.
We now assess the sensitivity of the explicit controller to the choice of γ over 20 Monte-Carlo realizations of the batch datasets D T , for different noise levels. This evaluation is performed by looking at the cost of the controller over the same noiseless closed-loop test of length T v = 50 considered previously, i.e., As shown in Figure 3, the value of γ that leads to the minimum closed-loop cost tends to decrease when the noise level increases, supporting our considerations in Section 4.4. At the same time, by properly selecting γ we attain a cost J = 16.37 ± 0.58, which is generally close to the oracle J O = 15.77, which is the one achieved by the oracle law, i.e., the model-based predictive controller obtained by exploiting the true model of S. These results additionally show that, for increasing levels of noise, the choice of γ becomes more challenging, since the range of values leading to the minimum J progressively shrinks. Note that, when γ is excessively small the optimal input is always zero and S evolves freely 6 .
We additionally evaluate the effect of averaging, by looking at the performance index J in (41) over 30 Monte-Carlo data-collections for an increasing number N of repeated experiments of length T = 100. The measurements are affected by noise, yielding SNR = 20 dB. Figure 4 shows that the use of the averaged datasetD T has a similar effect to a reduction of the noise level. Indeed, for increasing N the optimal γ slowly shifts towards smaller values, thus further implying the gradual reduction in the impact of γ on noise handling.

Altitude control of a quadcopter
As a final case study, we consider a nonlinear system, namely the problem of controlling the altitude of a quadcopter, to perform landing or take-off maneuvers. To this end, we exploit the same simulator used in [15]  These choices yield a dataset D T of length T = 800, that satisfies Assumption 1 and allows us to retain information on possible non-zero angular configurations.
The three attitude controllers introduced in [15] are further retained in testing to keep the attitude angles at zero and to decouple the altitude dynamics from that of the other state variables. Within this setting, the explicit data-driven law is designed by imposing L = 5, Q = P = diag([1, 0.1]), R = 10 −5 and γ = 1. To mitigate the effect of the gravitational force, the design and closed-loop deployment of the designed explicit controller are carried out by pre-compensating it. As a result, the input to be optimized is where m = 0.5 [kg] is the mass of the quadcopter, g = 9.81 [m/s] is the gravitational acceleration and u z (t) is the input prior to the compensation. A similar approach is adopted for the control problem to fit our framework γ ≤ 10 −3 when SNR = 40 dB, SNR = 20 dB and SNR = 10 dB, respectively.
(a) SNR =40 dB (b) SNR =20 dB (c) SNR =10 dB in both landing and take-off scenarios. We thus consider the reduced state wherez [m] is the altitude set point. To avoid potential crashes of the quadcopter, in designing the explicit law we impose the following constraint on the state of the system: which, in turn, guarantees the altitude to be always nonnegative. Meanwhile, the pre-compensated input is constrained to the interval: where the lower bound corresponds to a null input and the upper limit is dictated by the maximum power of the motors 7 .
The performance of the learned explicit law attained in take-off and landing are reported in Figure 5. Here we consider closed-loop tests in which the altitude and the vertical velocity are noisy, with the noise acting on the closed-loop measurements sharing the features of that corrupting the batch ones. Despite the noise acting on the initial condition at each step, both maneuvers are successfully performed, thus showing the effectiveness of the retrieved explicit data-driven laws.

Conclusions
By leveraging on the known PWA nature of the explicit MPC law within linear quadratic predictive control, in this paper we propose an approach to derive such an explicit controller from data only, without undertaking a full modeling/identification step. Thanks to the formalization of the problem, well-known model-based techniques can be straightforwardly adapted to check the stability of the closed-loop system before deploying the controller. Future research will be devoted to extend these preliminary results to cases in which the state is not fully measurable, to exploit priors to guarantee practical closedloop stability by design. Future work will also be devoted to formalize the connections between the explicit solution proposed in this paper and the one introduced in [8], consequently providing a comparative analysis of the two approaches.