{'fraction_non_alphanumeric': 0.053022399509051854, 'fraction_numerical': 0.015464866523473458, 'mean_word_length': 4.285760622770029, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Concurrent learning in high-order tuners for parameter identification

High-order tuners are algorithms that show promise in achieving greater efficiency than classic gradient-based algorithms in identifying the parameters of parametric models and/or in facilitating the progress of a control or optimization algorithm whose adaptive behavior relies on such models. For high-order tuners, robust stability properties, namely uniform global asymptotic (and exponential) stability, currently rely on a persistent excitation (PE) condition. In this work, we establish such stability properties with a novel analysis based on a Matrosov theorem and then show that the PE requirement can be relaxed via a concurrent learning technique driven by sampled data points that are sufficiently rich. We show numerically that concurrent learning may greatly improve efficiency. We incorporate reset methods that preserve the stability guarantees while providing additional improvements that may be relevant in applications that demand highly accurate parameter estimates at relatively low additional cost in computation.


I. INTRODUCTION
The problem of identifying the parameters in a linear parametric model through the use of online measurements of input-output data has been studied extensively from the standpoint of the continuous-time gradient algorithm, which plays an important role in adaptive control [1], [2] and has found broad applications in areas such as optimal nonlinear control [3], extremum seeking [4], and the analysis of machine learning algorithms [5]. The gradient algorithm has been shown to achieve uniform global asymptotic (and exponential) stability of the desired parameter value, under a condition of persistent excitation which has been shown to be both necessary and sufficient [6]. Here, the distinction between uniform and non-uniform asymptotic stability has crucial implications in practice: unlike the non-uniform notion of stability, the uniform notion ensures robustness, in the sense of achieving "total stability" in the presence of bounded additive disturbances ([7, Sec. 1B]) which are widely encountered in applications.
Although persistent excitation characterizes uniform asymptotic stability for the gradient algorithm, the incorporation of "concurrent learning" has been shown to enable uniform asymptotic stability in the absence of persistent excitation [8]. Concurrent learning involves an augmentation of the gradient, using discretely sampled data measurements that collectively satisfy a condition of "sufficient richness." Advantageously, sufficient richness can be characterized in terms of a rank condition on a matrix constructed from the data samples, a condition which can be computationally much simpler to verify in practice than the condition of persistent excitation.
In a separate thread of research, recent studies have shown that filtering the gradient can improve the performance of the gradient algorithm, sometimes significantly and in the absence of persistent excitation [9]. Namely, the use of filtering can improve the convergence rate of the parameter estimate toward the desired parameter value or, in adaptive control problems, the convergence rate of the tracking error toward zero. The filtering procedures considered in [9] and [10] give rise to algorithms referred to as high-order tuners, which take inspiration from algorithms introduced in [11]. In addition to offering improved performance in continuoustime settings, the recently developed high-order tuners serve as a basis for deriving novel discrete-time algorithms for various problems of estimation and learning with online data, with guarantees of efficiency in the sense of Nesterov's method for convex optimization [12] and of robustness in noisy and adversarial environments [13], [14]. In continuous time, under a persistent excitation condition, uniform asymptotic stability properties of high-order tuners can be established using the method of analysis in [15,Sec. 4.6]. However, the use of concurrent learning in high-order tuners has not yet been explored.
In this work, we establish uniform global asymptotic stability (UGAS) properties of continuous-time high-order tuners for parameter identification under two different conditions: (1) persistent excitation and (2) concurrent learning with sufficiently rich data. In Section III, UGAS is established under persistent excitation, using an approach that we claim to be simpler than that of [15,Sec. 4.6]. Whereas [15] shows uniform convergence by carefully examining solutions of a differential equation, we instead take advantage of a Matrosov theorem [16], which can be regarded as an analogue of the LaSalle invariance principle in the context of time-varying systems, with which uniform convergence is shown by combining infinitesimal conditions on Lyapunovlike functions together with observability-like conditions. In Section IV, we propose implementations of concurrent learning for high-order tuners in order to preserve UGAS in the absence of persistent excitation, given sufficiently rich data. We show that the resulting systems admit strict Lyapunov functions. In Section V, we propose the use of a technique inspired by reset methods in control and optimization ( [17], [18]), which shows promise in improving the efficiency of high-order tuners that make use of concurrent learning. In Section VI, numerical results show that concurrent learning can offer significant improvements in the convergence rate of high-order tuners for a parameter identification problem involving a regressor constructed from sinusoids.

II. NOTATION AND DEFINITIONS
For x ∈ R n , we use |x| := √ x T x. Given a pair (x, u) ∈ R n × R m , by abuse of notation we sometimes consider z := (x, u) to be a vector in R n+m . By the same abuse of notation, we sometimes write a function defined on R n × R m as a function defined on R n+m , i.e., f (x, u) may be written as f (z) with z = (x, u). A map ϕ : R n × R ≥0 → R m is said to be locally bounded in x uniformly in t if there exists a number M > 0 not dependent on t such that |ϕ(x, t)| ≤ M for all t and for all x within a sufficiently small ball centered at the origin.
For a locally bounded function f : is piecewise continuous, the origin of the systemẋ = f (x, t) is said to be uniformly globally stable (UGS) if there exists a class-K ∞ function γ such that, for each initial condition The origin is said to be uniformly globally attractive (UGA) if for each r > 0 and σ > 0 there exists The origin is said to be uniformly globally asymptotically stable (UGAS) if it is UGS and UGA. The origin is said to be uniformly globally exponentially stable (UGES) if there exist c > 0 and α > 0 such that, for each initial condition

HIGH-ORDER TUNERS VIA PERSISTENT EXCITATION
Suppose y * (t) = φ T (t)θ * for all t ≥ 0, where y * : R ≥0 → R and φ : R ≥0 → R n are known functions of time, and we wish to solve for θ * ∈ R n online under the following assumption.
Assumption 1: The regressor φ(·) is piecewise continuous, bounded, and persistently exciting. That is, there exist As a means of determining θ * , we follow the algorithmic development of [10], [9,Ch. 5] , and consider the following differential equation in the variable where β ∈ R >0 , γ ∈ R >0 , and t → N t are to be selected for the purpose of achieving desired convergence properties for solutions of (1). Choosing N t to be dependent on φ(t) will be crucial for establishing stability properties of (1). We focus on the case of although other choices may be feasible (see [19,Sec. 3] for an example to consider). Equation (1) is referred to as a high-order tuner and is identical to [10, Eq. 6] and [9, Eq. 5.6].
For brevity, the dependence of φ and e y on t will be suppressed hereafter.
Proof: Recalling thatθ = θ − θ * , let p := ϑ − θ and x := (θ, p). Reusing notation from the proof of Theorem 1, we will use Matrosov's theorem [16] to establish that the origin is UGAS for the systeṁ which will imply that (θ * , θ * ) is UGAS for (3). To begin, consider the Lyapunov function candidate (5), which is radially unbounded, positive definite, and continuously differentiable. Observing that the right-hand side of (9) can be obtained by multiplying the right-hand side of (4) by 1/N t , we have from (6) that, for all (x, t) ∈ R 2n × R ≥0 and for β ≥ 2γ/µ, and hence the origin of (9) is uniformly globally stable. Next, we establish uniform global attractivity by building Matrosov functions as follows. Let V 1 be defined as in (7) so that, following the steps leading up to (8), we may write, for all (x, t) ∈ R 2n × R ≥0 , and (x, t) → φ(t) are each locally bounded inx uniformly in t, and both Y 0 and Y 1 are continuous. Thus, the conditions of Matrosov's theorem [16] hold, and we conclude that the origin of (9) is UGAS. Theorems 1 and 2 also establish uniform global exponential stability (UGES), due to linearity of the systems (1)  Our assumptions differ from those of [10] only in regards to the regressor's properties. Namely, the analyses previously reported in [10, Thm. 2] and [10, Remark 8] require that the regressor has a bounded time derivative, whereas our analyses do not require differentiability of the regressor but instead require that it be persistently exciting. As a consequence, the previous analyses can establish only that the output error e y := φ Tθ tends to 0 and not necessarily uniformly, whereas Theorems 1 and 2 establish that the parameter errorθ tends to 0 uniformly.

A. Stability analysis
Let {(φ (t k ), y * (t k ))} N k=1 be a sequence of recorded data. Define B : R n × R ≥0 → R n as In (1), we implement concurrent learning (CL) in the sense of, e.g., [8], [21], and [19], as follows: where N t and µ are given by (2), and β and γ are positive real numbers. The B-term involves a factor of N t for reasons that will become clear in the stability analysis.
In (3), we implement CL as follows: where N t and µ are given by (2), and β and γ are positive real numbers. UGAS properties for (12) and (13) can be shown if the data satisfies the following property, which is characterized by the subsequent lemma.
Assumption 2: The regressor data {φ (t k )} N k=1 is sufficiently rich (SR) in the sense that the matrix Lemma 1: For a given µ ∈ R ≥0 , Assumption 2 holds if and only if there exists δ µ ∈ R >0 such that Proof: Let µ ∈ R ≥0 be given. First, we show the forward implication. Assuming that D has rank n, it follows that, for any non-zero x ∈ R n , there exists k ∈ {1, . . . , N } such that φ T (t k )x = 0. (If it were not true, the columns of D would not span R n .) In other words, for any non-zero x ∈ R n , there exists k such that x T φ(t k )φ T (t k )x > 0. It follows that, for any non-zero x ∈ R n , x T P µ x > 0, and hence there exists δ µ ∈ R >0 (which generally depends on µ) such that x T P µ x ≥ δ µ . Next, we show the reverse implication. Assuming that there exists δ µ ∈ R >0 such that P µ ≥ δ µ I n , it follows that, for any non-zero x ∈ R n , there exists k ∈ {1, . . . , N } such that x T φ(t k )φ T (t k )x > 0. That is, there exists k such that φ T (t k )x = 0. Then, because x is an arbitrary non-zero vector in R n , it follows that there are n linearly independent columns of D.
One benefit of the CL feature is that it can be implemented using only the sampled data, mitigating the practical costs that may be incurred by continuously collecting data online. For example, the following system can be interpreted as a novel high-order tuner which does not require continuous online measurement of φ(t) and y(t): In this case, an additional advantage is that the UGAS property can be established under no restriction on the relative magnitudes of β, γ, and µ.
As remarked in Section III, UGAS is equivalent to UGES for linear time-varying systems, and therefore Theorems 3, 4, and 5 also establish UGES, due to linearity of the systems (12), (13), and (18).

B. Online implementation
Standard analyses of stability and convergence for CL methods assume the availability of recorded (offline) data that satisfies a rank condition similar to Assumption 2 [22], [8], and the same assumption is made in Theorems 1-5. In the absence of such data, CL can be implemented via online criteria that aim to ensure that the rank condition is satisfied after executing the algorithm for some period of time. When evaluating the performance of high-order tuners in Section VI, the simple online criterion of [23, Sec. IV] is considered. Namely, the number of recorded data points at time t is denoted N (t), and a new data point is added to the data set k=1 at time t if the following condition is satisfied for some user-specified parameter ε ∈ R >0 : The data set is initialized as {(φ (t 1 ), y * (t 1 ))} with t 1 = 0 and N (0) = 1. The condition (20) is evaluated until N (t) reaches a user-specified maximum, an integer N ≥ n. Equation (11) is implemented in a time-dependent fashion, i.e., with N replaced by N (t), and it becomes time-independent after N (t) reaches N .
For this online implementation of CL, the stability analysis is beyond the scope of the current work and, to our knowledge, has not been pursued in previous studies of CL methods. Previous studies of online implementations have focused not on stability analysis of the resulting dynamics but on showing that, in certain circumstances, an online criterion ensures that a rank-condition is satisfied in finite time or that the convergence rate of the resulting dynamics is maximized after a rank-condition has been met [23].

V. SOFT-RESET METHODS FOR HIGH-ORDER TUNERS
For applications that demand high efficiency and precision, we explore the possibility that resetting the state ϑ−θ to zero under certain conditions can benefit performance, motivated by works such as [17] and [18]. Defining f as in (12), we incorporate resets in (12) via a soft-reset approach, similar to ideas proposed in [18] and [24], resulting in a differential inclusion given by where β r ∈ R >0 . The mapping 'SGN' is the set-valued sign mapping, i.e., SGN(s) is equal to s/|s| when s = 0 and SGN(0) = [−1, 1]. Consequently, when ϕ(x, t) < 0, (21) behaves like (12). On the other hand, when ϕ(x, t) > 0, the reset behavior becomes active, causing (21) to behave like (21) but with β increased to a value of β + 2β r . Based on these observations, a UGAS result for (21) can be obtained under the same conditions of Theorem 3, following similar steps in the proof and making use of Lyapunov conditions for differential inclusions. Defining f as in (13), soft resets are implemented in (13) according to the following differential inclusion: with β r ∈ R >0 . A UGAS result for (22) can be obtained using similar observations as those made above for (21), following similar steps as in the proof of Theorem 4 and making use of Lyapunov conditions for differential inclusions.
In contrast with the soft-reset approach, the authors of [19] have studied various hard-reset approaches in high-order algorithms for parameter identification.
VI. NUMERICAL RESULTS Figure 1 compares the efficiency of the basic gradient method given byθ and of the high-order tuner given by (1), along with their respective CL counterparts given bẏ θ = −γ (∇ θ L t (θ) + B(θ, 0)) and equation (12), using the online implementation described in Section IV-B with ε = 1 and N = 10. The soft-reset system is given by (21). Following [9, Sec. 5.7.1], we set φ(t) = [1, 1 + 3 sin(t), 1 + 3 cos(t)] T for all t ≥ 0 and randomly initialize the value of θ. We initialize ϑ at the same value as θ. For all algorithms, γ = 0.1, and µ = 0.2. For all high-order tuners, we set β = 1 to satisfy the requirements of Theorems 1 and 3. These parameter values are intended to match those chosen in [9,Sec. 5.7]. For the soft-reset system, β r = 4. All systems are implemented by Euler discretization with a stepsize of 10 −3 . Under the same conditions as in Figure 1, Figure 2 compares the efficiency of the basic normalized gradient method given byθ and of the high-order tuner given by (3), along with their respective CL counterparts given bẏ and equation (13). To satisfy the requirements of Theorems 2 and 4, the values of β, γ, µ, and β r are chosen to be the same as in the experiment shown by Figure 1.