{'fraction_non_alphanumeric': 0.04657831512051461, 'fraction_numerical': 0.009997674959311788, 'mean_word_length': 4.440134907251265, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Composite learning control with application to inverted pendulums

Composite adaptive control (CAC) that integrates direct and indirect adaptive control techniques can achieve smaller tracking errors and faster parameter convergence compared with direct and indirect adaptive control techniques. However, the condition of persistent excitation (PE) still has to be satisfied to guarantee parameter convergence in CAC. This paper proposes a novel model reference composite learning control (MRCLC) strategy for a class of affine nonlinear systems with parametric uncertainties to guarantee parameter convergence without the PE condition. In the composite learning, an integral during a moving-time window is utilized to construct a prediction error, a linear filter is applied to alleviate the derivation of plant states, and both the tracking error and the prediction error are applied to update parametric estimates. It is proven that the closed-loop system achieves global exponential-like stability under interval excitation rather than PE of regression functions. The effectiveness of the proposed MRCLC has been verified by the application to an inverted pendulum control problem.


I. INTRODUCTION
Adaptive control is one of the major control techniques of handling uncertainties in nonlinear systems and has still attracted great concern in recent years [1]- [16]. In particular, model reference adaptive control (MRAC) is a popular adaptive control architecture which aims to make an uncertain dynamical system behave like a chosen reference model. The way of parameter estimation in adaptive control gives rise to two different schemes, namely indirect and direct schemes [17]. Composite adaptive control (CAC) is an integrated direct and indirect adaptive control technique which aims to achieve better tracking and parameter estimation through faster and smoother parameter adaptation [18]. In the CAC, prediction errors are generated by identification models, and both tracking errors and prediction errors are applied to update parametric estimates. The superiority of CAC on performance improvement has been demonstrated in many control designs, where some latest results can be referred to [19]- [26]. Nevertheless, as in the classical adaptive control, CAC only achieves asymptotic convergence of tracking errors and does not guarantee parameter convergence unless plant states satisfy the condition of persistent excitation (PE) [17]. It is well known that the PE condition is very strict and often infeasible to monitor online in practical control systems [27].
Learning is one of the fundamental features of autonomous intelligent behavior, and it is closely related to parameter convergence in adaptive control [30]. The benefits brought by parameter convergence include accurate closed-loop identification, exponential stability and robustness against measurement noise [28]. An emerging concurrent learning technique provides a feasible and promising way for achieving parameter convergence without the PE condition in MRAC [27]- [29]. The difference between the concurrent learning and the composite adaptation lies in how to construct prediction errors. In the concurrent learning, a dynamic data stack constituted by online recorded data is used in constructing prediction errors, singular value maximization is applied to maximize the singular value of the data stack, and exponential convergence of both tracking errors and estimation errors is guaranteed if regression functions of plant uncertainties are excited over a time interval such that sufficiently rich data are recorded in the data stack. Yet in this innovative design, the singular value maximization leads to an exhaustive search over all recorded data, and the requirement on the derivation of all plant states for calculating prediction errors is stringent. This paper proposes a novel model reference composite learning control (MRCLC) strategy for a class of parametric uncertain affine nonlinear systems. In the composite learning, a modified modelling error that can utilize online recorded data is constructed as the prediction error, a second-order command filter is applied to alleviate the derivation of plant states, and both the tracking error and the prediction error are applied to update parametric estimates. It is proven that the closedloop system achieves global exponential-like stability under interval excitation (IE) rather than PE of regression functions. Consequently, the limitations of concurrent learning are alleviated by the proposed MRCLC design. The effectiveness and superiority of this approach is verified by the application to an inverted pendulum control problem with the comparison with some existing adaptive control approaches.
The notations of this paper are relatively standard, where R, R + , R n and R n×m denote the spaces of real numbers, positive real numbers, real n-dimensional vectors, and n × mdimensional matrixes, respectively, | · | and · denote the absolute value and Euclidean-norm, respectively, L ∞ denotes the space of bounded signals, Ω c := {x| x ≤ c} denotes the ball of radius c, min{·} and max{·} are the minimum and maximum functions, respectively, sgn(·) is the sign function, rank(A) is the rank of A, diag(·) is a diagonal matrix, and C k represents the space of functions whose k-order derivatives all exist and are continuous, where c ∈ R + , x ∈ R n , A ∈ R n×m , and n, m and k are positive integers.

II. PROBLEM FORMULATION
This section discusses the formulation of learning from the classical MRAC. For clear illustration, consider a class of nth order affine nonlinear systems as follows [27]: A reference model that characterizes the desired response of the system (1) is given bẏ is the vector of reference model states, and r(t) ∈ R is a bounded reference signal. This study is based on the facts that is linearly parameterizable such that [27]- [29] f in which W * ∈ Ω cw ⊂ R N is a unknown constant parameter vector, Φ(x) : R n → R N is a known regression function vector, and c w ∈ R + is a known constant. The following definitions are given for facilitating control synthesis.
Let x re := [x T r , r] T be an augmented reference signal, and W ∈ R N be an estimate of W * . Define a tracking error e(t) := x(t) − x r (t), and an estimation errorW (t) := W * −Ŵ (t). Our objective is to design a proper parametric update law of MRAC such that both e andW exponentially converge to 0 under the IE rather than the PE conation.

A. Review of Previous Results
From [17], the MRAC law can be designed as follows: where u pd denotes a proportional-derivative (PD) feedback part, u re denotes a reference signal feedforward part, u ad denotes an adaptive part, k e ∈ R n and k r ∈ R n+1 are control gains, and the design of k r satisfies Thus, one obtains the tracking error dynamicṡ where A := Λ − bk T e is designed to be strictly Hurwitz. Thus, for any matrix Q = Q T > 0, a unique solution P = P T > 0 exists for the following Lyapunov equation: Let the adaptive law ofŴ be as follows: where γ ∈ R + is a learning rate, and P(Ŵ , •) is a projection operator given by [17] P(Ŵ , Choose a Lyapunov function candidate with z := [e T ,W T ] T ∈ R n+N for the closed-loop dynamics composed of (6) and (8). It follows from the classical MRAC result of [17] that ifŴ (0) ∈ Ω cw and Φ(x) is of PE, then the closed-loop system (6) with (8) achieves global exponential stability in the sense that both the tracking error e(t) and the estimation errorW (t) converge to 0.
To remove the requirement of the PE condition on Φ(x) for parameter convergence in [17], a concurrent learning law ofŴ is proposed in [27] as follows: where j is a certain epoch, p ≥ N is the number of stored data, and ǫ j :=W T j Φ(x j ) denotes a modelling error which is regarded as the prediction error calculated by where x j , u j andŴ j are online recorded data of x, u andŴ at the epoch j, respectively. Let Z := [Φ(x 1 ), Φ(x 2 ), · · · , Φ(x p )] ∈ R p×N be a dynamic data stack. It is shown in the concurrent learning MRAC approach of [27] that ifŴ (0) ∈ Ω cw and rank(Z) = N on t ≥ T e , then the closed-loop system (6) with (10) achieves global exponential-like stability in the sense that both the tracking error e and the estimation errorW converge to 0 on t ≥ T e . Yet in the approach of [27], fixedpoint smoothing must be applied to estimateẋ j in (11) such that the prediction error ǫ j is calculable, and singular value maximization should be applied to exhaustively search the data stack Z to maximize its singular value.

B. Composite Learning Control Design
Define a modified modelling error ǫ(t) := Θ eW (t) as the prediction error, in which with σ = max t≥0 {σ r (t)} and in which τ d ∈ R + is an integral duration, σ r (t) ∈ R + is the minimal singular value of Θ(t), and T e ≥ τ d is the time when σ r (t) reaches its maximum 1 . Then, a composite learning law ofŴ is designed as follows: in which k w ∈ R + is a weight factor.
From ǫ = Θ eW and (12), the calculation of ǫ can follow the calculation of ΘW as follows: where ΘŴ is directly obtainable by (13) and (14), and ΘW * is not directly obtainable. Multiplying both sides of (6) by Φ(x(t)), integrating the resulting equality over [t − τ d , t] and making some transformations, one gets in which the time variable τ is omitted in the above integral part. Sinceė n is unavailable, a second-order linear filter with unit gain is implemented as follows [31]: withê n (0) = e n (0) andê n+1 (0) = 0, where ω ∈ R + is the natural frequency, ζ ∈ R + is the damping factor, andê n and e n+1 are estimates of e n andė n , respectively. The integral in (16) effectively reduces the influence of measurement noise on the calculation of ΘW * . Thus, ω in (17) can be made sufficiently small so thatê n+1 ≈ė n . For simplifying analysis, assume thatê n+1 =ė n as in [31]. The reasonability of this assumption will also be verified in the subsequent simulations.
The following theorem shows the main result of this study.
Theorem 1: Consider the system (1) driven by the control law (4) with (14), where the control gain k r is designed to satisfy (5), and the control gain k e is selected to make A in (6) strictly Hurwitz. IfŴ (0) ∈ Ω cw and Θ(T e ) ≥ σI are satisfied with c w , σ ∈ R + and T e ≥ τ d , then the closed-loop system composed of (6) and (14) achieves global exponential-like stability in the sense that all closed-loop signals are bounded for all t ≥ 0 and both the tracking error e(t) and the estimation errorW (t) exponentially converge to 0 on t ≥ T e .
Secondly, consider the control problem at t ∈ [T e , ∞). since there exist σ ∈ R + and T e ≥ τ d such that Θ(T e ) ≥ σI, i.e. the bounded signal Φ(x(t)) is exciting over t ∈ [T e − τ d , T e ], it is obtained from (19) thaṫ It follows from (9) and (20) thaṫ where k s := min{λ min (Q)/λ max (P ), 2γk w σ} ∈ R + , which implies that the closed-loop system has global exponential-like stability in the sense that both e andW exponentially converge to 0 as long as t ≥ T e .
Simulations are carried out in MATLAB software running on Windows 7, where the solver is set to be fixed-step ode 5 with the step size being 0.001 s and other settings being default values. The conventional MRAC in [17] and the model reference CAC (MRCAC) with Q-modification in [32] are selected as baseline controllers, where the reference models and shared parameters of all controllers are set to be the same for fair comparisons. Simulation trajectories by all controllers are depicted Fig. 1. For the control performance, it is shown that the plant state x follows its desired signal x r closely with a smooth control input u for all controllers, the MRCAC achieves the worst tracking accuracy [see Fig. 1(c)], and the proposed MRCLC achieves the best tracking accuracy [see Fig. 1(e)]. For the learning performance, it is observed that IE occurs with σ rising at t = 5 s in this case, the MRAC does not show any parameter convergence [see Fig. 1(b)], the MRCLC shows better parameter estimation than the MRAC but still does not achieve parameter convergence [see Fig. 1(d)], and the proposed MRCLC achieves fast and accurate parameter estimation even the IE is short and weak [see Fig. 1(f)].

V. CONCLUSION
In this paper, a MRCLC strategy has been successfully developed for a class of parametric uncertain affine nonlinear systems such that parameter convergence can be guaranteed by the IE rather than PE condition. The proposed approach has also been applied to an inverted pendulum model, where superior control and learning performances have been demonstrated compared with the conventional MRAC and the MRCAC with Q-modification. Further work would focus on the extension of the composite learning to wider classes of nonlinear systems such as multi-input multi-output affine nonlinear systems [34] and strict-feedback nonlinear systems [35].