{'fraction_non_alphanumeric': 0.04524077175434169, 'fraction_numerical': 0.018445020532703204, 'mean_word_length': 4.0445550283532, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 15, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Adaptive Systems with Closed-loop Reference Models: Stability, Robustness and Transient Performance

This paper explores the properties of adaptive systems with closed-loop reference models. Using additional design freedom available in closed-loop reference models, we design new adaptive controllers that are (a) stable, and (b) have improved transient properties. Numerical studies that complement theoretical derivations are also reported.


I. INTRODUCTION
The central element of any adaptive systems is online parameter adjustment. This is usually accomplished by having a plant, determined by a dynamic model, along with a controller with adaptive parameters designed to compensate for the plant's actions, follow a reference model. The resulting error between the reference model and the plant is used to adjust the adaptive parameter.
By definition, open-loop reference models are independent of the system dynamics. Such reference models have been the backbone of adaptive control for the past four decades [1], [2] where modifications to the adaptive control law were first added for stability in the presence of bounded disturbances [3]- [5] and semi-global stability in the presence of unmodeled dynamics [6], [7]. We denote the underlying open-loop systems in all these cases as Open-loop Reference Model (ORM)adaptive systems.
Earlier developments of adaptive systems included explorations of various kinds of reference models. The overall goal behind the selection of a reference model is that the corresponding tracking error must asymptotically decay in the absence of parametric uncertainties in the plant being controlled. In order to accomplish this goal, modifications of the open-loop reference models were explored [8], [9]. Some of these modifications retained stability properties and were otherwise indistinguishable from ORM-adaptive systems and as a result, not pursued. Others could not be shown to be stable and were therefore dropped. Recently, a class of Closed-loop Reference Models (CRM) have been proposed for control of plants with unknown parameters whose states are accessible (see for example [10]- [14]) all of which are guaranteed to be stable and additionally portray improved transient performance.
Transient performance has been directly addressed in [16] and more recently in [11], [13]- [15] . The results in [11] discussed the tracking error, but focused the attention mainly on the initial interval where the CRM-adaptive system exhibits fast time-scales. Results in [13], [14] focus on deriving a damping ratio and natural frequency for adaptive systems with CRM. However, assumptions are made that the initial state error is zero and that the closed-loop system state is independent of the feedback gain in the reference model, both of which may not hold in general. The results in [15] too assume that initial state errors are zero. And in addition, the bounds derived in [15] are based upon L ∞ norms, which do not capture the transient properties of adaptive systems. The results in [16] pertain to transient properties of adaptive systems, and quantify them using an L 2 norm. The adaptive systems in question however are indirect, and do not pertain to CRMs. With the exception of [13], [14], none of the others have focused on derivatives of signals in the adaptive system, which is another measure of transient performance.
Our focus in this is paper on CRM-based adaptive systems. Similar to [11], [13], [14] we demonstrate their stability properties. Unlike these papers, we discuss transient properties of these adaptive systems using an L 2 norm of error signals and derivatives of key signals such as adaptive parameters and the control input. These metrics are used to compare the CRM adaptive systems with their ORM counterparts Another class of adaptive systems that have been explored in the past where a noticeable improvement in transient performance was obtained is in the context of Combined/composite direct and indirect Model Reference Adaptive Control (CM-RAC) [17], [18]. While the results of these papers established stability of combined schemes, no rigorous guarantees of improved transient performance were provided, and have remained a conjecture [19]. We focus on this class of adaptive systems as well in this paper and introduce CRMs into the picture. The resulting CMRAC with Closed-loop reference models (CMRAC-C) are shown to be stable, enable the feedback of noise-free state estimates while guaranteeing stability, and most importantly are shown to have guaranteed transient properties.
The main contributions of the paper are (i) direct adaptive control structures with guaranteed transient performance, (ii) combined direct and indirect adaptive controllers with guaranteed transient performance, and (iii) the development of adaptive systems that allow feedback from noise free regressors. These are realized by using the extra degree of freedom available in the CRM in terms of a feedback gain, and by exploiting exponential convergence properties of the CRMadaptive system. The latter is made possible by introducing a projection algorithm with a known upper bound on the unknown parameters.
Previous work by the same authors on this subject can be found in [21]- [23]. The findings presented in [21] are preliminary and illustrate the waterbed affect through Euclidean bounds of the rate of control input at specific times of interest. The results presented in [22], [23] are a condensed version of this article and do not contain all of the discussions or proofs necessary to have a clear self contained presentation of this material. While much of what is presented in the paper is restricted to plants with state-variables accessible, the same idea can be extended to adaptive control using output feedback and is the topic of current investigation [20].
The results in this paper are organized as follows: Section II introduces the basic structure of CRM adaptive control as well as the Projection Operator. Section III investigates the transient response of CRM. Section IV investigates the robustness properties of CRM adaptive control. Section V contains the stability analysis of CMRAC-C. Section VI analyses the transient performance of CMRAC-C. Section VII contains the analysis of CMRAC-CO and its robustness properties in regard to measurement noise. Section VIII compares CRM, CMRAC-C and CMRAC-CO structures. Section IX contains our concluding remarks.

II. THE CRM-ADAPTIVE SYSTEM
In this section, we describe the CRM-adaptive system, and establish its stability and convergence properties in the absence of any perturbations other than parametric uncertainties. We first describe the CRM-adaptive system and prove its closedloop stability. After some preliminaries on matrix bounds, we introduce a projection algorithm in the adaptive law. This is used to derive exponentially converging bounds on the key variables in the CRM-adaptive system. Consider the linear system dynamics with scalar inpuṫ where x(t) ∈ R n is the state vector, u(t) ∈ R is the control input, A p ∈ R n×n is unknown and b ∈ R n is known. Our goal is to design the control input such that x(t) follows the reference model state x m (t) ∈ R n defined by the following dynamicṡ where A m ∈ R n×n is Hurwtitz and r(t) ∈ R is a bounded possible time varying reference command. L ∈ R n×n is denoted as the Luenberger-gain, and is chosen such that is Hurwitz. Equation (2) is referred to as a CRM, and when L = 0 the classical ORM is recovered.

Assumption 1.
A parameter vector θ * ∈ R n exists that satisfies the matching condition The control input is chosen in the form where θ(t) ∈ R n is the adaptive control gain with the update lawθ is the model following error and P = P T > 0 is the solution to the algebraic Lyapunov equationĀ T m P + PĀ m = −I n×n .
The underlying error model in this case is given bẏ whereθ(t) = θ(t) − θ * is the parameter error.
Proof: It is straight forward to show using (6) and (8)  that V (e,θ) = e T P e +θ T Γ −1θ (9) is a Lyapunov function. Since e is bounded, the structure of (2) implies that x m is bounded. x in turn and u are bounded. Barbalat lemma ensures asymptotic convergence of e(t) to zero.
The above corollary seems redundant. It is however used later in the transient performance.
The overall CRM-adaptive system is defined by (1), (2), (5), and (6). The standard open-loop reference model is given byẋ with the corresponding tracking error One can in fact view the error e o as the true tracking error and e as a pseudo-tracking error. The question that arises is whether the convergence properties that are assured in an ORM-adaptive system, of e o (t) tending to zero is guaranteed in a CRM-adaptive system as well. This is addressed in the following corollary: Proof: From Theorem 1 we can conclude that e(t) → 0 asymptotically. Thus we can conclude that

Remark 1.
The choice of the CRM as in (2) essentially makes the reference model nonlinear, as x depends on θ which in turn depends on x m in a highly nonlinear manner. In general, the tracking problem in a ORM-adaptive system can be viewed as one where an overall nonlinear time-varying system is to be designed such that its output tracks that of a linear timeinvariant system. The CRM-adaptive system is one where the overall nonlinear system is instead required to follow a nonlinear (reference) model. This nonlinear model, however, is chosen such that it asymptotically approaches the original linear reference model in the classical case, and hence the CRM-adaptive system retains all the desired characteristics of the ORM-adaptive system. As we will show in Section III, the CRM-adaptive system has an additional desirable property, of quantifiable transient properties, which the ORM-adaptive system may not necessarily possess. We will also show in this section that this is made possible by virtue of the additional degree of freedom available to the adaptive system in the form of the feedback gain in the CRM.

A. Preliminaries
All norms unless otherwise noted are the Euclidean-norm and the induced Euclidean-norm. The variable t ∈ R + denotes time throughout and for a differentiable function x(t), d dt x(t) is equivalent toẋ(t). Parameters explicit time dependence (t) is used upon introduction and then omitted thereafter except for emphasis. The other norms used in this work are the L 2 and truncated L 2 norm defined below. Given a vector ν ∈ R n and finite τ 0 ν(s) p ds 1/p and the infinity norm is then defined as ν(t) L∞ sup ν(t) .

Definition 1. Given a Hurwtiz matrix
For ease of exposition, throughout the paper, we choose L in (2) and Γ in (17) as follows: Proof: A m is Hurwitz and therefore σ > 0. It is not necessary however that the sum A m + A T m is Hurwitz. The trace operator is denoted as Tr(·) and is a linear operator. Recalling that a matrix and its transpose have the same trace we can conclude that Σ i λ i (A m ) = Σ i λ i (A m +A T m )/2. Finally we have that

B. Projection Algorithm
Before we evaluate the benefits of closed-loop reference models, we introduce a modification in the adaptive law to ensure robustness properties.

Assumption 2.
A known θ * max exists such that θ * ≤ θ * max . The projection based adaptive law, which replaces (6), is given byθ where the Γ-projection function, Proj Γ , is defined as in Appendix A and f is a convex function given by where ϑ and ε are positive constants chosen as ϑ = θ * max and ε > 0.

Definition 2.
Using the design parameters of the convex function f (θ; ϑ, ε) we introduce the following definitions θ max ϑ + ε and C. Convergence of the Adaptive System Theorem 6. Let Assumptions 1 and 2 hold. Consider the adaptive system defined by the plant in (1) with the reference model in (2), the controller in (5), the adaptive tuning law in (17) and L and Γ as in (13)- (14). For any initial condition in e(0) ∈ R n , and θ(0) such that θ(0) ≤ θ max , e(t) and θ(t) are uniformly bounded for all t ≥ 0 and converge exponentially to a set E through V in (9) aṡ where and E (e,θ) e 2 ≤ β 1θ 2 max , θ ≤θ max with Proof: See Appendix C.

D. CRM free design parameters
In this section it is argued that the free design parameters for the adaptive system are ℓ and the ratio of ℓ and γ. It is clear from (20) that the rate of exponential decay of the Lyapunov function is solely a function of the slowest eigenvalue of σ and ℓ. The term σ is defined by the open-loop reference model jacobiam A m in (2) and cannot be independently increased without also increasing the bound on θ * , due to the fact that A m = A p + bθ * T from (4). Therfore, from (21) it is clear that while increasing σ increases α 1 it also increases the size of of the compact set E that the model following error is exponentially converging to. The Luenberger gain ℓ, in contrast, does not affect the matching condition for the adaptive system and therefore increasing ℓ does not result in a high-gain matching condition. It is seen from (22) that while increasing ℓ may result in a larger α 1 which is desirable, it also increases the α 2 and β 1 . At first glance this seems undesirable. However, it is important to note that ℓ is inversely proportional to the bounds for P in (15) and (16). Upon further inspection of the tuning law in (17), when Projeciton is not active the adaptive law reduces toθ = −γxe T P b, and therefore, the only fixed parameters that control the rate of adaptation are γ, P (ℓ) and b. Given that b is fixed it is ignored and thus the learning rate is a function of γ P . Therefore, if γ and ℓ are increased at the same rate, the effective rate of change in the adaptive tuning law will remain the same, which illustrates that the ratio of ℓ and γ is an important design parameter. We therefore introduce ρ, an effective learning rate, Using the definition of ρ the bounds from Theorem 6 can be rewritten as This reparameterization in terms of ρ and ℓ is used for discussing the transient performance.

SYSTEMS
In the following subsections we derive the transient properties of the CRM-adaptive systems. Five different subsections are presented, the first of which quantifies the Euclidean and the L 2 -norm of the tracking error e. In the second subsection we compute the same norms for the parameter derivativeθ(t).
In both cases, we show that the L 2 -norms can be decreased by increasing ℓ. In the third theorem, we address the performance of the true error e o and show its dependence on ℓ. In the fourth subsection, we define our metric for transient performance in terms of a truncated L 2 norm of the rate of control effort. The last subsection compares ORM and CRM adaptive systems using these metrics.
A. Bound on e(t) Theorem 7. Let Assumptions 1 and 2 hold. Consider the adaptive system defined by the plant in (1) with the reference model in (2), the controller in (5), the adaptive tuning law in (17) and L and Γ as in (13) and (14).
Proof: see Appendix D.

B. Bound onθ(t)
In addition to e(t) L2 we explicitly compute upper bounds for θ (t) and θ (t) L2 . From the definition ofθ(t) in (17), it follows that We note that x(t) = e(t) + x m (t) and from (2) and (89) that Using the bound for e(t) L2 from (17) and the Cauchy-Schwartz inequality, we simplify (27) as (28) The above bounds make the following theorem possible. Theorem 8. Let Assumptions 1 and 2 hold. Consider the adaptive system defined by the plant in (1) with the reference model in (2), the controller in (5), the adaptive tuning law in (17) and L and Γ as in (13) and (14).
, and the a i and b i are independent of ρ and ℓ.
Proof: see Appendix E.

C. Bound on e o (t)
Here, we derive a bound on the true error e o (t) defined in (10).

Theorem 9. Let the assumptions from Theorem 8 hold. The difference between the open-loop reference model and the closed loop reference model satisfy the following bound
Proof: see Appendix G

D. Bound onu(t)
We now derive a final transient measure of the CRMadaptive system that pertains tou. This is chosen as the transient performance metric because the rate of change of the control authority requested by the controller directly affects the robustness of the system to unmodelled dynamics and actuator rate limits. Before the bounds are derived, several variables must be defined.

Definition 3.
Let time-constants τ 1 (ℓ), τ 2 be defined as Let constants a θ and δ 1 (ℓ, N ) be defined as where N > 0, and three intervals of time  (2) and δ 1 (ℓ) will be used in the following Lemma to formally define our time scale separation condition. The time interval T 1 is the time interval over which e(t) decays by N time constants, T 3 is the asymptotic time scale for e(t) and T 2 is an intermediate time interval. We note that T 1 exists but is unknown.
Remark 3. The condition Lemma 10 (i) defines the time scale separation condition. Recall that τ 1 is the time scale associated with e(t) and a θ is an upper bound on the uncertain open-loop eigen values of the plant. When ℓ ≥ ℓ * we are able to show that at t N = N τ 1 , e(t N ) has exponentially decade by N time constants, while x(t N ) has not deviated far from x(0).

Remark 4.
The bound onṙ(t) is needed so thatu(t) is well defined. The analysis techniques that follow in proving Theorem 11 will still hold for reference inputs with discontinuities. The metric for transient performance however would change fromu to d dt θ T (t)x(t) . Assumption 4. For ease of exposition we will assume that x m (0) = 0.
We note that the same analysis holds for x m (0) with addition of exponentially decaying terms proportional to x m (0). Theorem 11. Let Assumptions 1-4 hold. Given arbitrary initial conditions in x(0) ∈ R n and θ(0) ≤ θ max , for any ǫ > 0, N > 0 and ℓ ≥ ℓ * ,u satisfies the following inequalities: where ǫ 1 exp(−N ) and the κ i are independent of ρ and ℓ, and N ≥ 3 Proof: see Appendix H.

Remark 5.
There are two "small" terms in the above analysis. ǫ 1 is determined by the number of time constants N of interest. ǫ is free to choose and from Corollary 2 proves the existence of a finite T and is used to define when T 3 begins.
From Theorem 11, it follows that where c i > 0, i = 1 to 8 are independent of ℓ and ρ, L 1 (·) and L 2 (·) are globally lipschitz with respect to their arguments. The inequalities in (37) lead us to the following three main observations (see Figure 1) (A1) Over T 1 , |u(t)| is bounded by a linear function of ρ and √ ρ, (A2) Over can be made small by choosing a large ℓ. There is a penalty, however, in choosing a large ℓ, as the bound G x,2 increases linearly with ℓ. Therefore, after choosing an ℓ which satisfies the time scale separation as needed in Lemma 10, a ρ (which through (23) defines a choice for γ) can be chosen such that the integral in the following theorem is minimized.
Proof: u(ρ, ℓ) L2,τ is continuous with respect to ρ and ℓ where ρ and ℓ appear in the numerator of (37) and are positive. Therefore, ρ opt and ℓ opt exist and are finite.
τ in Theorem 12 denotes the interval of interest in the adaptive system where the transient response is to be contained. Given that T , and therefore T 1 is a function of ℓ, (38) can only be minimized over T 1 ∪ T 2 . From the authors definition of smooth transient performance in the beginning of this section choosing ρ opt and ℓ opt will guarantee smooth transient performance.

E. Comparison of CRM and ORM-adaptive systems
The bounds on e(t) and the L 2 -norm ofθ directly show that CRM-adaptive systems lead to smaller e(t) than with the ORM which are obtained by setting ℓ = 0 in (25) and (26). However, the same cannot be said for either e o or for the Euclidean norm ofθ; for a non-zero ℓ, the bound on e o is larger than that of e. This indicates that there is a tradeoff between fast transients and true tracking error. The signal that succinctly captures this trade off isu, whose behavior is captured in detail using the time intervals T1, T2, and T3. We also showed in Theorem 12 that this trade-off can be optimized via a suitable choice of ℓ and ρ. In what follows, we compare this optimized CRM with ORM and show that the former is clearly better than the latter.

Definition 4. The following two time constants
are used to describe the three time intervals that will be used in the analysis ofu for the ORM case where As in Definition 3, here too, T exists but is unknown. While these periods for both CRM and ORM are indicated in Figure  1, one cannot apriori conclude if T 1 is greater than or smaller than T ′ 1 . The time instants indicated as in Figure 1 are meant to be merely sketches.
For the adaptive system with the classical MRAC given by Eqs (1), (2), (5), (17)-(18) and (13)-(14) with ℓ = 0, it can be shown that sup are independent of ρ 0 , and M 1 (·) and M 2 (·) are globally lipschitz with respect to their arguments The proof of Proposition 13 follows the same steps as in the proof of Theorem 11 and is therefore omitted.
The bounds in (42) indicate that in the classical ORM, one can only derive a bound foru over the period T ′ 1 , T ′ 2 and T ′ 3 . Unlike the CRM case, the procedure in Appendix H cannot be used to derive satisfactory bounds foru over [N τ * 1 , N τ ′ 2 ). It also can be seen that unlike the CRM case, τ ′ 2 is fixed and cannot be changed with ℓ. These points are summarized below. (B1) Over T ′ 1 , |u(t)| is bounded by a linear function of ρ 0 and √ ρ 0 (B4) τ ′ 2 is fixed and unlike τ 1 , can not be adjusted. We now compare the bounds onu using observations (A1)-(A3) and (B1)-(B3). In order to have the same basis for comparison, we assume that γ, σ, and ℓ are such that ρ = ρ 0 and that both CRM-and ORM-adaptive systems start with the same bound at t = 0. As noted above, a tight bound cannot be derived for the ORM-based adaptive system over [N τ * 1 , N τ ′ 2 ). In the best scenario, one can assume that this bound is no larger than that over [0, N τ * 1 ]. This allows us to derive the bounds shown in Figure 1. The main observations that one can make from this figure are summarized below: • Even though at time t = 0, both the ORM and CRM have the same bound, since τ 1 can be made much smaller than τ ′ 2 , this bound is valid for a much shorter time with the CRM-system than in the ORM-system. This helps us conclude that the initial transients can be made to subside much faster in the former case than the latter, by suitably choosing ℓ.
• The bound onu for T 2 with the CRM-adaptive system is however linear in powers of ℓ and hence can be larger than the bound onu with the ORM-adaptive system over T ′ 2 . • The above observations clearly illustrate, if the cost function U (N τ ′ 2 ; ρ, ℓ) is minimized then the CRM system will have smoother transients than the ORM. Then, at larger times the error dynamics will asymptotically converges to zero.

F. Water-Bed Effect
The discussions in the preceding sections clearly show that CRM-adaptive systems introduce a trade-off: a fast convergence in e(t) with a reduced θ (t) L2 occurs at the expense of an increased e o (t). While an optimal choice of ρ and ℓ can minimize this trade-off, it also implies that a badly chosen ℓ and ρ can significantly worsen the adaptive system performance in terms of e o (t) andu(t). We denote this as the water-bed effect and illustrate it through a simulation. Consider a first-order plant with a single unknown parameter, whose values are identical to the example in Section IV-A over the first ten seconds. Figure 2 shows the behavior ofu(t) for the ORM, the optimized CRM, and a poorly chosen CRM. The plots clearly show the water-bed effect for the last case and the improved performance of the optimized CRM over the ORM. The free design parameters are also shown in the figure.
IV. ROBUSTNESS OF CRM TO TIME-VARYING UNCERTAINTIES AND DISTURBANCES We now evaluate the CRM-adaptive system in the presence of perturbations due to time-varying parameters and disturbances. Consider the uncertain Linear Time Varying systeṁ where d(t) is a bounded disturbance and A p (t) is time varying with a bounded time-derivative. It is assumed that a timevarying vector θ * (t) exists such that and θ * d , θ max exist where θ * (t) ≤ θ * d and θ * (t) ≤ θ max . Theorem 14. With Assumptions 1 and 2, consider the adaptive system defined by the plant in (43) with the reference model in (2), the controller in (5), the adaptive tuning law in (17) and L and Γ as in (13)- (14). For any initial condition in e(0) ∈ R n , and θ(0) such that θ(0) ≤ θ max , e(t) and θ(t) are uniformly bounded for all t ≥ 0 and the Lyapunov candidate in (9) converges exponentially to a set E aṡ Proof: See Appendix F.

Remark 7.
From the above Theorem it is is shown that in the presence of disturbance and time-varying uncertainty, the rate of exponential decay of the lyapunov function is α 1 /2 where α 1 was the rate of exponential decay for the system with constant uncertainty and no disturbances. The term that controls how the compact set E grows with the upper bound on the rate of change of the time-varying matched uncertainty is β 2 which is inversely proportional to γ. β 3 scales the size of the compact set in regard to the disturbance term d. Therefore, increasing γ and ℓ in proportion so that ρ is constant constant decreases the size of the compact set E.

A. Simulation Study
For this study a scalar time varying system of the form in (43) is controlled where is a deterministic signal used to represent a disturbance. Over the first 20 seconds d(t) = 0. After 20 seconds d(t) is generated from a Gausian distribution centered at 0 with a variance of 1, covariance of 0.1, deterministically sampled at 10 Hz with a fixed seed, and then passed through a saturation function with upper and lowers bounds of 0.2 and -0.2 respectively. The reference model to be followed is defined asẋ with control input from (5) and the update law for the adaptive parameter defined (6), where ℓ and ρ are chosen as in Table  I. The simulations have three distinct regions of interest, with Region 1 denoting the first 10 seconds, Region 2 denoting the 10 sec to 20 sec range, and Region 3 denoting the 20 sec to   35 sec range. In Region 1, the adaptive system is subjected to non-zero initial conditions in the state and the reference input is zero. At t = 10 sec, the beginning of Region 2, a filtered step input is introduced. At t = 20 sec In Region 3, timevariations in the plant parameter as well, disturbances and a filtered step input are introduced. Figures 2 and 3 illustrate the response of the CRM-adaptive system over 0 to 35 seconds, with x m , x, and e indicated in Figure 2, and u,u, and θ indicated in Figure 3. In both cases, the resulting performance is compared with the classical adaptive system. The first point that should be noted is a satisfactory behavior in the steadystate of the CRM-adaptive controller. In particular, as can be seen from the latter half of Region 1, both e and e o tend to zero as t approaches 10 seconds. The same satisfactory trends are observed in regions 2 and 3 as well, underscoring the robustness property of CRM-adaptive control, which validates Theorem 2.
We also note yet another significant difference between the responses of CRM-adaptive control and the classical one, which pertains to the rate of control inputu. An examination of Regions 2 and 3 clearly illustrates that the control input is smoother for CRM-adaptive control.

V. CMRAC
We now return to CMRAC introduced in [17] and [18]. We will show that the introduction of a CRM in these adaptive systems not only ensures stability, but also enables stability with observer-based rather than state-based feedback. In addition, the use of a CRM in CMRAC enables the derivation of transient properties which could not be accomplished hitherto. Section A addresses stability of the CMRAC with CRM, denoted as CMRAC-C, in the scalar case. Section B extends the results from section A to higher order plants with states accessible and addresses transient properties of the CMRAC-C adaptive system. Section C introduces an additional feature of observer feedback. Denoting the underlying adaptive system as CMRAC-CO, it is shown that the resulting adaptive system has guaranteed stability properties and results in reduced error bounds in the presence of measurement disturbances with a zero mean property, which is corroborated through simulations in Section D. Section E contains extensions to higher order plants whose states are accessible.

A. Stability of CMRAC-C
We assume that the plant and reference model dynamics are given by Equations (1) and (2) with A m and L = L m satisfying Equations (4) and (3). The control input is chosen as in (5) and the identifier dynamics are given bẏ where L i is Hurwitz. The error dynamics are now given bẏ The update laws for the adaptive parameters are then defined with the update laẇ where ǫ θ = θ(t) −θ(t), with Γ chosen as in (14), η > 0, and P m = P from (7) and P i = 1 2(σ+ℓ) I n×n . Theorem 15. Let Assumptions 1 and 2 hold. Consider the overall CMRAC-C specified by (1), (2), (5), (49), (50) and (52). For any initial condition e m (0), e i (0) ∈ R n , and θ(0) andθ(0) such that θ(0) ≤ θ max and θ (0) ≤ θ max , it can be shown that e m (t), e o (t), θ(t) andθ(t) are uniformly bounded for all t ≥ 0 and the function converges exponentially to a set E aṡ Proof: see Appendix J.

Remark 8.
There is no appreciable difference between the CMRAC-C and CRM adaptive controller presented in Section II in terms of stability and the bounds for the set E.
VI. TRANSIENT PROPERTIES OF CMRAC-C In the following subsections we derive the transient properties of the CMRAC-C adaptive system, similar to what was done in Section III. Two different subsections are presented, the first of which quantifies the Euclidean and the L 2 -norm of the tracking error e and the second subsection, were the truncated L 2 norm of the rate of control effort is presented.
Proof: see Appendix K.

(62)
with ǫ 1 = exp(−N ) Proof: The finite time stability result used in (107) still holds for the MMRAC-C. Therefore G ′′ x,1 in (62) is identical to G x,1 in (36). The Lyapunov function in (53) has two additional terms in e i andθ as compared to the Lyapunov equation in (9). Therefore, G ′′ e,1 now includes the initial conditions of the estimation error e i (0). G ′′ x,2 and G ′′ e,2 are similarly affected. Barbalat Lemma can be used for G ′′ e,3 , and G ′′ x,3 follows from the same analysis in Appendix H-C. The η terms arise from the righthand side of the update law in (52).

VII. CMRAC-CO
When measurement noise is present, it is often useful to use a state observer for feedback rather than the plant state. However, the use of such an observer in adaptive systems has proved to be quite difficult due to the inapplicability of the separation principle. In this section, we show how the CRM can be used to avoid this difficulty for a class of plants. We denote the resulting adaptive system as CMRAC-CO.
We assume that the plant and reference model dynamics are given by Equations (1) and (2) with A m and L = L m satisfying Equations (4) and (3). The control input is now chosen as u = θ T (t)x o + r (63) and x o is the state of the observer dynamics, given bẏ Defining e m (t) = x(t) − x m (t) and e o (t) = x o − x(t), the error dynamics are now given bẏ For ease of exposition we choose The update laws for the adaptive parameters are then defined with the update laẇ with Γ chosen as in (14), η > 0, with P from (7).
converge exponentially to a set E aṡ Proof: see Appendix L.

A. Robustness of CMRAC-CO to Noise
As mentioned earlier, the benefits of the CMRAC-CO is the use of the observer state x o rather than the actual plant state x. Suppose that the actual plant dynamics is modified from (1) aṡ where n(t) represents measurement noise. For ease of exposition, we assume that n(t) is bounded and deterministic. This leads to a set of modified error equationṡ and E (e m , e o ,θ,θ) e m 2 ≤ β 6θ 2 max + β 7 n(t) 2 , e o 2 ≤ β 6θ 2 max + β 7 n(t) 2 , θ ≤θ max , θ ≤θ max with β 6 defined in (72) and β 7 defined as Proof: see Appendix M

B. Simulation Study
For this study a scalar system in the presence of noise is to be controlled with dynamics as presented in (73), where A p = 1, b = 1, and n(t) is a deterministic signal used to represent sensor noise. n(t) is generated from a Gausian distribution with variance 1 and covariance 0.01, deterministically sampled using a fixed seed at 100 Hz, and then passed through a saturation function with upper and lower bounds of 0.1 and -0.1 respectively. The reference model, identifier and observer are from (2), (49) and (64) respectively, with A m = −1 and b = 1. The controller is defined by (67). The design parameters for the two test cases are shown in Table II. The simulations have two distinct regions of interest, with Region 1 denoting the first 4 seconds, Region 2 denoting the   Open θx + r θxo + r 4 sec to 15 sec range. In Region 1, the adaptive system is subjected to non-zero initial conditions in the state and the reference input is zero. At t = 4 sec, the beginning of Region 2, a filtered step input is introduced. Figures 5 and 6 illustrate the response of the CMRAC-CO adaptive system over 0 to 15 seconds, with x m , x, and e m indicated in Figure 5, and u, u, θ andθ indicated in Figure 6. In both cases, the resulting performance is compared with the classical CMRAC system. The first point that should be noted is a satisfactory behavior in the steady-state of the CMRAC-CO adaptive controller. We note a significant difference between the responses of CMRAC-CO and CMRAC systems, which pertains to the use of noise free regressors in CMRAC-CO. An examination of ∆u/∆t in Figure 6 clearly illustrates the advantage of CMRAC-CO.

VIII. COMMENTS ON CMRAC, CMRAC-C AND CMRAC-CO
As discussed in the Introduction, combining indirect and direct adaptive control has always been observed to produce desirable transient response in adaptive control. While the above analysis does not directly support the observed transient improvements with CMRAC, we provide a few speculations below: The free design parameter L i in the identifier is typically chosen to have eigenvalues faster than the plant that is being controlled. Therefore the identification model following error e i converges rapidly andθ(t) will have smooth transients. It can be argued that the desirable transient properties of the identifier pass on to the direct component through the tuning law, and in particular ǫ θ .
The CMRAC-C differs from classical CMRAC only due to the Luenberger gain L m in the reference model. Given the contributions of Section III which show that the CRM can result in satisfactory transients without the indirect component raises the question if the added complexity of a CMRAC-C is justified. One answer to this question is in the form of the CMRAC-CO, where it is shown that one can design stable observer-based feedback in a CMRAC, allowing noise-free estimation and control.

IX. CONCLUSION
This paper concerns the introduction of a feedback gain L in the reference model and the analysis of various adaptive systems with this feature. In particular, we show that with closed-loop reference models (CRM), (i) direct adaptive control structures result in guaranteed transient performance, (ii) combined direct and indirect adaptive controllers result in guaranteed transient performance, and (iii) observer-based feedback can be used in adaptive systems while retaining stability. These are primarily realized using the extra degree of freedom available in the CRM in terms of a feedback gain, and by exploiting exponential convergence properties of the CRM-adaptive system. In all cases, a projection algorithm is used in the adaptive law with a known upper bound on the unknown parameters.
The main impact of this work is the quantification of transient performance in adaptive systems through L 2 norms of tracking errors and the control input derivativeu. It is shown that the introduction of the feedback gain L introduces two time-scales to govern the adaptive system dynamics. The first has to do with the convergence of the tracking error, and the second has to do with adaptation to the unknown plant parameter. By allowing these two time-scales to be separate, transients in the adaptive systems can be controlled without compromising learning of the unknown parameter. This in turn is accomplished by choosing L in an optimal manner. Sub-optimal choices can result in better transients in e only at the expense of slow adaptation leading to a "water-bed" effect. This paper, to our knowledge, is the first to illustrate this effect via an exhaustive formal and experimental analysis of CRM-based adaptive systems.

APPENDIX A PROJECTION OPERATOR
The Γ-Projection Operator for two vectors θ, y ∈ R k , a convex function f (θ) ∈ R and with symmetric positive definite tuning gain Γ ∈ R k×k is defined as where ∇f (θ) = ∂f (θ) ∂θ1 · · · ∂f (θ) ∂θ k T . The projection operator was first introduced in [24] with extensions in [2] and for a detailed analysis of Γ-projection see [25]. Definition 6. The following compact sets will be referred to in the following analysis: Before we prove the above theorem, we introduce the following two lemmas.

Theorem 24.
Giveṅ where η > 0 is a scaler, θ * ∈ D 0 , θ(0) ∈ D 1 ,θ(0) ∈ D 1 and f is convex Proof: Given that θ andθ both begin in D 1 either both parameters hit the boundary of D 1 simultaneously or only one parameter is at the boundary of D 1 while the other is strictly inside. Lets consider the case where θ(t) is on the boundary of D 1 and thus f (θ) = 1 andθ ⊆ D 1 . Consider the quadratic function F (θ) as first presented in (84). Differentiating F (θ) and using the update law in (86) we havė From (85) we already know that the first part of (88) is less than 0. For the second part, given that f (θ) is convex and sinceθ ∈ D 1 , (∇f (θ)) T (θ −θ) ≥ 0, and thereforeḞ (θ) ≤ 0.

Proof of Lemma 5(i):
Beginning Using the bound in (89) and (90) the integral just above can be upper bounded and the bound in (15) directly follows.

Proof of Lemma 5 (ii):
Let ξ ∈ R n be a normalized eigenvector of P . By pre-and postmultiplying (7) by ξ T and ξ, we have ξ TĀT m P ξ + ξ T PĀ m ξ = −ξ T I n×n ξ which reduces to Finally, using the definitions in (12) and taking the minimum eigenvalue of P we arrive at (16) [28].

APPENDIX C PROOF OF THEOREM 6
Proof: Recall the Lyapunov candidate in (9), Taking its time derivative one has thaṫ Using the upper bound on P from (15) with α 1 defined in (21) and α 2 σ+2ℓ m 2 γθ 2 max . Using the Gronwall Bellman Inequality, (91) implies that Thus, e exponentially converges to the set defined by the following inequality lim t→∞ e(t) T P e(t) ≤ 1 γθ 2 max .
Using the bound in Lemma 5(ii) we have that then we can conclude that lim t→∞ e(t) 2 ≤ β 1θ 2 max where β 1 is defined in (22). The boundedness of θ(t) follows from Theorem 21.

APPENDIX D PROOF OF THEOREM 7
Proof: From (92) and (93), we know that Using the following inequalities the fact that θ (0) ≤θ max and the definition of ρ from (23), the result in (25) holds with Beginning with using the definitions of ρ from (23) and the fact that 1 σ+2ℓ ≤ 1 σ+ℓ the bound in (26) holds.

APPENDIX E PROOF OF THEOREM 8
Proof: Using (15), the choice for Γ in (14) and the definition of ρ from (23) we have that Γ P ≤ m 2 ρ. Using the bounds in (28) and (25) for x m (t) and e(t) the results in (29) follow immediately.
For the L 2 norm we begin by observing that Taking the supremum of (28) and (25) we have upper bounds for sup x m (t) 2 and sup e(t) 2 . The L 2 norm of e(t) is given in (26).

APPENDIX F PROOF OF THEOREM 14
Proof: Taking the time derivative of the Lyapunov candidate in (9), substitution of the update law from (17) and the plant dynamics in (43), the derivative of the lyapunov function can be upper bounded aṡ After completing the square in e and d we havė and then neglecting the negative quantity after −1/2 e 2 , Writing the above inequality in terms of the Lyapunov candidate in (9) we havė (97) Using the upper bound on P from (15) and rewriting (97) in terms of the design parameters γ and ℓ we havė where α 3 is defined just before α 4 in (46). Following the same procedure as in Appendix C we conclude that lim t→∞ e T P e ≤ 1 γθ 2 max + 4m 2 θ * (σ + 2ℓ)γθ max + 2 (99) Recalling the fact that s ≥ σ > 0 from Lemma 4 we can conclude that 2(s + ℓ) σ + 2ℓ ≤ 2s σ . (100) Using the bound above along with that in (93) the inequality in (99) can be simplified as where β 1 is defined in (22), and β 2 and β 3 are defined in (47). The boundedness of θ(t) follows from Theorem 21.

APPENDIX G PROOF OF THEOREM 9
Proof: The dynamics of the CRM and the ORM are given in (2) and (10) respectively and leed to the followinġ Given that the reference model will have the same initial condition regardless of being closed or open, we then have that where the matrix exponential bound came from (89). Using the Cauchy-Schwartz inequality we have the following bound APPENDIX H PROOF OF THEOREM 11 Taking the time derivative of u in (5) Substitution of the upper bound on P from (15), using the definition of a θ from (33) and the bounds on the reference trajectory from Assumption 3 results in the following bound A. Proof of Theorem 11, t ∈ T 1 The following Lemma is useful:
Proof: Suppose z(t) ∈ R is defined as the solution tȯ It can be shown that if z(0) = x(0) , then using [29,Theorem 8.14].
C. Proof of Theorem 11, t ∈ T 3 G e,3 follows from Corollary 2. G x,3 follows from (115), where it is noted that t ≥ N τ 2 , and the fact that x ≤ e + x m

APPENDIX I PROOF OF THEOREM 13
The bound given for x(t) and e(t) over the time period [0, N τ * 1 ] in (107) holds regardless of the choice of ℓ. Thus the bound in (35) holds for ℓ = 0, and therefore the bound in (42) for t ∈ [0, N τ * 1 ] is the same as that in (37) where ρ has been replaced by ρ 0 . The Gronwall-Bellman analysis used to obtain the bound for e(t) would follow with a similar bound to that in (111) where the exponent would now have the time constant τ ′ 2 with e(t) exponentially decaying to 1/ρ 0θmax . For t > N τ ′ 2 , e(t) would have decayed past 4 time constants. Therefore, the coefficient G e,2 would apply for the ORM case when t > N τ ′ 2 and ℓ = 0. The bound for x(t) would not contain the parameter ℓ. Therefore, (42) for t > N τ ′ 2 is identical in structure to (37) for t > N τ 1 with ℓ = 0 and ρ being replaced with ρ 0 . The asymptotic properties of the adaptive system hold regardless of the choice of ℓ and therefore Corollary 2 holds when l = 0 as well and thus the bounds in (42) for t ≥ T ′ 1 hold as well. 16

APPENDIX J PROOF OF THEOREM 15
Proof: Taking the time derivative of V in (53) results iṅ Substitution of V in (53) results in the bound in (54). Using the bound in Lemma 5-(ii) we have that e T m P m e m ≥ 1 2(s + ℓ) e m 2 and e T i P i e i ≥ 1 2(σ + ℓ) e i 2 then we can conclude that lim t→∞ e m (t) 2 ≤ β 4θ 2 max and lim t→∞ e i (t) 2 ≤ β 5θ 2 max where β 4 and β 5 are defined in (55). The boundedness of θ(t) andθ(t) follows from Theorem 24. The asymptotic limit to zero comes from the application of Barbalat Lemma.

APPENDIX K PROOF OF THEOREM
The bounds in (56) and (57) follow from the application of Gronwall-Bellman to the result in (54) with the lower bound for min λ i (P ) in (16) and the change of parameters from (23) being used.
Neglecting the negative terms in lines 2 and 3 from the equation above and substitution of the norm for P we have thatV (1 − ∆(ℓ)) 2 P 2 n(t) 2 .