{'fraction_non_alphanumeric': 0.048851519564094135, 'fraction_numerical': 0.022875663488186388, 'mean_word_length': 4.268497896560257, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Designing Experiments for Data-Driven Control of Nonlinear Systems

In a recent paper we have shown that data collected from linear systems excited by persistently exciting inputs during low-complexity experiments, can be used to design state- and output-feedback controllers, including optimal Linear Quadratic Regulators (LQR), by solving linear matrix inequalities (LMI) and semidefinite programs. We have also shown how to stabilize in the first approximation unknown nonlinear systems using data. In contrast to the case of linear systems, however, in the case of nonlinear systems the conditions for learning a controller directly from data may not be fulfilled even when the data are collected in experiments performed using persistently exciting inputs. In this paper we show how to design experiments that lead to the fulfilment of these conditions.


INTRODUCTION
Recent advances in learning have prompted a renewed interest in the use of data for control of complex dynamical systems in at least two different ways. On one hand, the use of new learning-based identification techniques for system identification followed by off-the-shelf top-notch robust control design (Chiuso, 2016;Recht, 2019). On the other hand, the design of control policies directly form data, skipping altogether any attempt of identifying the system's model (Campi et al., 2002;Campi and Savaresi, 2006;Formentin et al., 2013;Novara et al., 2016;Tanaskovic et al., 2017). Related contributions are the iterative feedback tuning (Hjalmarsson and Gevers, 1998), correlation-based tuning (Karimi et al., 2007), and the design of controllers for all the systems compatible with the measured data via polynomial optimization (Dai and Sznaier, 2018). Optimal (Gonçalves da Silva et al., 2019;Baggio et al., 2019) and nonlinear (Wabersich and Zeilinger, 2018) control problems have also been investigated.
Inspired by the the so-called Fundamental Lemma of Willems et al. (2005), a novel approach to simulate and control unknown dynamical systems without any identification, but merely using finite-length input-output data, was introduced in Markovsky and Rapisarda (2008). The spotlight on the ideas of Markovsky et al. (2005); Markovsky and Rapisarda (2008) and their important role in data-driven control design was turned on again by the recent paper by Coulson et al. (2019a). See Coulson et al. (2019b) and Huang et al. (2019) for recent follow-ups.
Also relying on the results of Willems et al. (2005), recently De Persis and Tesi (2019b,a) have shown that data collected from linear systems excited by persistently exciting inputs during low-complexity finite-horizon off-line experiments, can be used to design state-and outputfeedback controllers, including optimal Linear Quadratic Regulators (LQR), by solving linear matrix inequalities (LMI) and semidefinite programs. The authors have also shown how these methods are robust to the use of data corrupted by bounded-but-unknown deterministic noise, meaning that they return a stabilizing controller for noisy data that satisfy a quantified signal-to-noise ratio.
In a continuation of De Persis and Tesi (2019b,a), the necessity of the persistency-of-excitation conditions has been throughly discussed in van Waarde et al. (2020), the design of controllers robust to process disturbances has been further explored in Berberich et al. (2019) and databased guarantees for polyhedral set-invariance properties (safe controllers) have been studied in Bisoffi et al. (2019).
Another result of De Persis and Tesi (2019a) was to extend the stabilization result to the case of nonlinear systems (see Theorem 1 in Section 2 below). Namely, if the system is nonlinear and if the data collected during a finite-length off-line experiment satisfy suitable conditions (Assumptions 1 and 2 below) then we can design a controller from these data that stabilizes the system in the first approximation. In contrast with what happens with linear systems, however, where it can be shown that these conditions are always satisfied if persistently exciting inputs are applied in the experiment, in the case of nonlinear systems the fulfilment of Assumptions 1 and 2 is a more challenging task.
The purpose of this note is threefold. First we show that experiments that are performed on nonlinear systems may fail to generate data that satisfy Assumptions 1 and 2, even when persistently exciting inputs are applied. Second, we show how to design experiments that lead to the fulfilment of these assumptions. Third, we provide some heuristic considerations on how to verify that the stabilizing controller that we design is actually stabilizing when no a priori knowledge about the nonlinear system is available.
In Section 2 we recall the results of De Persis and Tesi (2019a). The main results are in Section 3. Conclusions are drawn in Section 4.

PRELIMINARIES
We start by recalling the result given in De Persis and Tesi (2019a).
Consider a smooth nonlinear system (1) and let (x, u) be a known equilibrium pair, that is such that x = f (x, u). Throughout his note, for ease of notation we will assume without loss of generality that the equilibrium point of interest is (x, u) = (0, 0).
Let us rewrite the nonlinear system as The quantity d accounts for higher-order terms and it has the property that it goes to zero faster than x and u, namely we have with R(x, u) a matrix of smooth functions with the property that It is known that if the pair (A, B) defining the linearized system is stabilizable then the controller K rendering A + BK stable exponentially stabilizes the equilibrium of the nonlinear system. Let ] be data resulting from an experiment carried out on the nonlinear system (1). Note that the matrices X 0,T , X 1,T and U 0,T are available from data.
Consider the following assumptions.
Assumption 1. The matrices have full row rank.
The following result holds.
The reason for considering the maximization of α is to render condition γ < α 2 /(4 + 2α) easier to fulfil. Notice that (7) is a semidefinite program.
Theorem 1 rests on the fact that Assumptions 1 and 2 hold and that (7) admits a solution with γ < α 2 /(4 + 2α). For linear controllable systems, this is always the case provided that the experiments are carried out with persistently exciting inputs of a sufficiently high order (see (De Persis and Tesi, 2019a, Definition 1) for a definition of persistency of excitation). In fact, under these conditions: (i) Assumptions 1 and 2 are satisfied; (ii) problem (7) is feasible, and any solution is such that γ < α 2 /(4 + 2α) (since D 0,T = 0 Assumption 2 holds with an arbitrary γ). We summarize this fact. Theorem 2. (De Persis and Tesi (2019a)). Let (1) be a linear controllable system with state-space dimension n, and consider an experiment carried out with a persistently exciting input of order n + 1. Then, Assumptions 1 and 2 hold. Further, problem (7) is feasible and any solution is such that γ < α 2 /(4 + 2α). Thus, any solution returns a stabilizing state-feedback gain In the sequel, it will be useful to relate the evolution of the nonlinear system with the one of the corresponding linearized system, which we express as We also let 1) · · · u l (T − 1) be the data resulting from an (hypothetical) experiment made on the linearized system (8). Note that the matrices X l 0,T , X l 1,T and U l 0,T are not available from data.

MAIN RESULTS
Since around an equilibrium point nonlinear systems behave as linear systems, it is tempting to conclude that Theorem 2 can be extended to the nonlinear case as long as the experiments are carried out sufficiently close to the equilibrium point. As we will see, this is true only if the experiments are carried out in a certain manner. More precisely, we will see that there indeed exist experiments that ensure the same properties as in Theorem 2, but not all experiments (even with persistently exciting inputs and arbitrarily close to the equilibrium) guarantee these properties. In the remainder of this section: (1) we show that there exist experiments (carried out with persistently exciting inputs and arbitrarily close to the equilibrium) for which Assumptions 1 and 2 fail to hold; (2) we show that there exist experiments (carried out with persistently exciting inputs and sufficiently close to the equilibrium) ensuring the same properties as in Theorem 2, and we characterize a class of experiments ensuring these properties; (3) we provide some heuristic considerations on how to verify that a solution to (7) returns a stabilizing controller. (In general, one cannot determine how close to the equilibrium the experiments should be run unless we have some prior knowledge on D 0,T , that is on the type of nonlinearity; thus, in general one cannot assess the fulfilment of the condition γ < α 2 /(4 + 2α) since γ is unknown.)

Experiment design issues for nonlinear systems
We start with showing that there exist experiments (carried out with persistently exciting inputs and arbitrarily close to the equilibrium) for which Assumptions 1 and 2 fail to hold. We show this fact through an example.
Example. Consider the nonlinear system with equilibrium (x, u) = (0, 0). The system can be written as with A = 0 and B = 1, and the corresponding linearized system is given by Consider now the experiment given by The input is persistently exciting of order n+1 = 2. Now, If we hypothetically apply this experiment to the linearized system, that is we let x l (0) = x 0 and U l 0,T = U 0,T , we obtain which is full row rank for all θ = 0, and the same holds for X l 1,T . However, has rank 1 for all θ = 0. We conclude that Assumption 1 does not hold. Note that this holds true even though the input sequence if persistently exciting and θ is arbitrarily small.
This simple example shows that for nonlinear systems there might exist experiments for which Assumption 1 is not satisfied, even if these experiments originate from persistently exciting inputs and are carried out arbitrarily close to the equilibrium.
An intuitive explanation for this fact can be obtained by relating the matrices associated with the linear and nonlinear system. Consider an experiment on the nonlinear system with initial condition x 0 and input sequence {u (0), u(1), . . . , u(T − 1)} resulting in state matrices X 0,T and X 1,T . It is simple to see that This implies that the matrices in Assumption 1 can be written as 10) and (11) In connection with the previous example, this translates to This means that there exist trajectories (i.e. experiments) for which the perturbation that causes to lose rank is of the same order (θ 2 in this example) as the nonlinear terms, thus of the same order as Ξ. In the sequel, we will show that one can nonetheless design experiments so as to satisfy Assumptions 1 and 2.

Scaling the experiments ensures Assumptions 1 and 2
The basic idea is to show that there also exist experiments such that Ξ vanishes faster than the perturbation causing the matrix (12) to lose rank.
In order to prove the first of (14) it is sufficient to show that X 0,T = εX l 0,T +Ξ since the relationŨ 0,T = εU l 0,T holds by construction. The result can be proved by induction on the entriesx of the matrixX 0,T . The claim holds for k = 0. Suppose that the claim holds up to a certain k ≥ 0. We havẽ which gives the claim.
The important fact resulting from these relations is that the matrices X l 0,T and X l 1,T are fixed, and the perturbation which cause them to lose rank depends on ε. To this end, we recall the following result.
Theorem 3. (Dahleh et al., 2003, Theorem 5.1) Suppose M ∈ C m×n is full column rank. Then, min ∆∈C m×n { ∆ | M + ∆ has rank < n} = σ n (M ) where · denotes the Euclidean norm, and where σ n is the smallest singular value of the matrix M .
The reasoning for full row rank matrices is analogous (one can alternately consider (14) with the transpose).
Recalling that the singular values of a matrix M are the square roots of the eigenvalues of the matrix M M , in connection with (14) it follows that the smallest perturbation that causes εM with to lose rank is of order ε. In fact, since M is full row rank then the smallest eigenvalue of the matrix M M is strictly positive. Denoting this eigenvalue by λ, it follows that the smallest singular value of εM is ε √ λ. Hence, to prove that the perturbed matrix in (14) is full row rank, it is sufficient to show that the matrixΞ goes to zero faster than ε since this implies the existence of a value ε such that Ξ < ε λ ∀ε ∈ (0, ε) This fact can be proven by looking at the various elements d which form the entries of the matrixΞ, since with the right-hand side being the Frobenius norm Ξ 2 F . In fact, if one can prove thatd converges to zero faster than ε, then the bound (16) on the maximum allowable perturbation holds.
Claim. For all k ≥ 0,d(k) converges to zero faster than ε.
Proof of the Claim. Recall that, for all k, with R(x(k),ũ(k)) a matrix of smooth functions with the property that For k = 0, we have hence, by (18), lim ε→0 R(x(0),ũ(0)) = 0, i.e. R(x(0),ũ(0)) converges to zero as fast as ε. The two just established facts and (17) written for k = 0 allow us to conclude thatd(0) goes to zero faster than ε. We now proceed by induction.
Similar arguments can be used to show Assumption 2. To this end, recall from (14) thatX 1,T = εX l 1,T +Ψ, where X l 1,T has full row rank. Thus, by applying the Young's inequality we obtaiñ Hence, a sufficient condition for Assumption 2 to hold is that The result follows by noting that X l 1,T does not depend on ε and is full row rank, while the entries ofΨ andD 0,T are functions ofd which goes to zero faster than ε (meaning thatΨΨ andD 0,TD 0,T converge to zero faster than ε 2 ).
We summarize the results in the following theorem. Note that, by an abuse of notation, in the statement below as well as in the subsequent Theorem 5, when we refer to Assumptions 1 and 2 and to problem (7), we let the matrices U 0,1,T , X 0,T , X 1,T , D 0,T therein to be replaced by the matricesŨ 0,1,T ,X 0,T ,X 1,T ,D 0,T .
Theorem 4 gives a principled method to satisfy Assumptions 1 and 2. In fact, if one can perform multiple experiments on the system at the equilibrium (x 0 = 0), one can apply scaled versions of the input sequence u [0,T −1] until Assumptions 1 and 2 are satisfied. In connection with the example of Section 3.1, an intuitive explanation of Theorem 4 is given in Figure 1.
We can further strengthen the result.
Proof. Point (i) has been previously shown.
Regarding the second of (7), first notice that the above LMI remains satisfied if we scale it by δ > 0, that is X 0,TQ − δαX 1,TX 1,TX 1,TQ Q X 1,TX 0,TQ 0,Q := δQ (23) for all δ > 0. Consider now the matrix which is well defined sinceX 0,TQ 0. Exploiting the relationQ := δQ, we obtain 0 for sufficiently small δ. By the Schur complement, this shows the feasibility of the second of (7), which together with (23) concludes the proof of (ii).
(iii). Here we exploit the fact that in the design formulation we search for the solution maximizing α. Let (Q, α) be the solution to (7) associated to the experiment (x 0 , u [0,T −1] ) carried out on the linearized system, with corresponding matrices X l 0,T and X l 1,T . Consider the scaled experiment (εx 0 , εu [0,T −1] ) still on the linearized system, resulting in state matrices εX l 0,T and εX l 1,T . Denote its solution by (Q, α). It follows that α = α, that is the optimal value of α does not change. To see this fact, we first show that (Q, α) = (εQ, α) is feasible. For the first of (7), the scaled experiment (εx 0 , εu [0,T −1] ) gives since (Q, α) is a feasible solution to (7) associated with the experiment (x 0 , u [0,T −1] ).
Concerning the second of (7), the scaled experiment (εx 0 , εu [0,T −1] ) gives since (Q, α) is a feasible solution to (7)  We finally show that for the scaled experiment we cannot have a solution with α > α. Consider the solution (Q, α) associated with the scaled experiment (εx 0 , εu [0,T −1] ) and suppose by contradiction that α > α. We show that in this case (Q,α) := (Q/ε, α) is a solution associated with the experiment (x 0 , u [0,T −1] ), contradicting the fact that α is the optimal value. For the first of (7) the solution (Q, α) associated with the scaled experiment (εx 0 , εu [0,T −1] ) gives From the relation (Q,α) = (Q/ε, α) we would thus get showing that (Q,α) withα > α is a solution to the first of (7) for the experiment (x 0 , u [0,T −1] ). Similarly, for the second of (7) the solution (Q, α) associated with the scaled experiment (εx 0 , εu [0,T −1] ) gives From the relation (Q,α) = (Q/ε, α) we would thus get To summarize, for the linearized system all the experiments (εx 0 , εu [0,T −1] ) result in the same optimal value for α. The proof is concluded noting that for the nonlinear system, the state matrices X 0,T and X 1,T associated with the experiment (εx 0 , εu [0,T −1] ) converge to εX l 0,T and εX l 1,T , respectively, as ε converges to zero, and this implies that the optimal value of α for the nonlinear system converges to α. In turn, this implies that for ε sufficiently small the solution returns a stabilizing controller since: (a) α is fixed; and, as stated in Theorem 4, (b) for any given γ, Assumption 2 is satisfied for ε sufficiently small.

Practical considerations
The foregoing analysis also gives a simple method to assess whether the solution returns a stabilizing controller. Specifically, if we have some prior knowledge on D 0,T , that is on the type of nonlinearity, then we can directly assess the fulfilment of the condition γ < α 2 /(4 + 2α) by computing the smallest value of γ that satisfies Assumption 2. If instead we have no prior knowledge on D 0,T , one can perform multiple experiments on the system, each one being an ε-scaled version of the experiment (εx 0 , εu [0,T −1] ). In this case, it follows from the foregoing analysis that α converges (faster than linearly) to the optimal fixed value α associated with solution for the linearized system. We illustrate these considerations through a numerical example. Consider the Euler discretization of an inverted pendulum x 1 (k + 1) = x 1 (k) + ∆x 2 (k) x 2 (k + 1) = ∆g sin x 1 (k) + 1 − ∆µ m 2 x 2 (k) + ∆ m 2 u(k) where we simplified the sampled times k∆ in k, with ∆ the sampling time. In the model m is the mass to be balanced, is the distance from the base to the center of mass of the balanced body, γ is the coefficient of rotational friction, g is the acceleration due to gravity. The states x 1 , x 2 are the angular position and velocity, respectively, u is the applied torque. The system has an unstable equilibrium Stability achieved γ < α 2 /(4 + 2α) fulfilled linear system 0 0 YES YES (ε-invariant) nonlinear system 0.0082 2e-7 YES YES (ε = 1) nonlinear system 0.0020 5e-8 YES YES (ε = 0.5) nonlinear system 0.0001 2e-9 YES YES (ε = 0.1) nonlinear system 1e-5 3e-10 YES YES (ε = 0.01) Table 1. Some tests for the inverted pendulum system.
The first experiment is generated starting at the equilibrium (x 0 = 0), with random input sequence within the interval [−5, 5], which causes a displacement of about 28 degrees from the equilibrium position. All the other experiments are generated starting at the equilibrium and by scaling the input sequence of a factor ε. The solution for the linearized system results in α = 0.01136 and K = [−56.9163 − 12.6186]. (Notice that we allow for such large input sequences since we start from x 0 = 0.) As the examples shows, α converges to α very quickly. In this respect, an interesting point observed in simulations is that also K converges to K. In fact, simulations indicate that the solution Q for the nonlinear system with experiment (εx 0 , εu [0,T −1] ) converges to εQ, where Q is the solution for the linearized system with experiment (x 0 , u [0,T −1] ). This property immediately gives that K = U 0,1,T Q(X 0,T Q) −1 converges to εU 0,1,T εQ(εX l 0,T εQ) −1 = K.

CONCLUSIONS
We have studied the conditions under which a controller that stabilizes unknown nonlinear systems in the first approximation directly from experimental data can be designed. We have shown via an example that these conditions may not hold even when the data have been obtained applying persistently exciting inputs. This is in sharp contrast with the results derived for linear systems. Nevertheless, for those experiments for which the conditions for the existence of a stabilizer of the nonlinear system do not hold, we have shown that a suitable scaling of the initial conditions and the applied input leads to the fulfilment of these conditions, and hence they allow the design of a data-based stabilizer of the unknown nonlinear system. We regard these results as a principled method to design stabilizers in the first approximation and will also have an impact on the nonlocal design of stabilizers for nonlinear systems, which we are currently investigating. Important practical aspects have been neglected. The scaling of the experiments may make the obtained data more sensitive to noise. Although the approach of De Persis and Tesi (2019a) allows for a robust analysis of the effect of noise on the controller design, this effect has not been studied in the current paper and deserves attention.