{'fraction_non_alphanumeric': 0.036697631600970795, 'fraction_numerical': 0.011381705582057075, 'mean_word_length': 4.354918216446337, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 16, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Concurrent learning for parameter estimation using dynamic state-derivative estimators

A concurrent learning (CL)-based parameter estimator is developed to identify the unknown parameters in a linearly parameterized uncertain control-affine nonlinear system. Unlike state-of-the-art CL techniques that assume knowledge of the state-derivative or rely on numerical smoothing, CL is implemented using a dynamic state-derivative estimator. A novel purging algorithm is introduced to discard possibly erroneous data recorded during the transient phase for concurrent learning. Since purging results in a discontinuous parameter adaptation law, the closed-loop error system is modeled as a switched system. Asymptotic convergence of the error states to the origin is established under a persistent excitation condition, and the error states are shown to be ultimately bounded under a finite excitation condition. Simulation results are provided to demonstrate the effectiveness of the developed parameter estimator.

the closed-loop system and the performance of the developed controller critically depend on convergence of the parameter estimates to their ideal values.
Data-driven concurrent learning (CL) techniques are developed in results such as [8], [17], [18], where recorded data is concurrently used with online data to achieve parameter convergence under a relaxed finite excitation condition as opposed to the persistent excitation (PE) condition required by traditional adaptive control methods.
CL techniques employ the fact that a direct formulation of the parameter estimation error can be obtained provided the state-derivative is known or its estimate is otherwise available through techniques such as fixed-point smoothing [19]. The parameter estimation error can then be used in a gradient-based adaptation algorithm to drive the parameter estimates to their ideal values. If exact derivatives are not available, the parameter estimation error can be shown to decay to a neighborhood of the origin provided accurate estimates of the state-derivatives are available, where the size of the neighborhood depends on the derivative estimation error [19]. Experimental results such as [8] demonstrate that, since derivatives at past data points are required, noncausal numerical smoothing techniques can be used to generate satisfactory estimates of state-derivatives. Under Gaussian noise, smoothing is guaranteed to result in the best possible linear estimate corresponding to the available data [20,Section 5.3]; however, in general, the derivative estimation error resulting from numerical smoothing can not be quantified a priori. Furthermore, numerical smoothing requires additional processing and storage of data over a time-window that contains the point of interest. Hence, the problem of achieving parameter convergence under relaxed excitation conditions without using numerical differentiation is also motivated.
In this paper, an observer is employed to estimate the state-derivative. The derivative estimate generated by the observer converges exponentially to a neighborhood of the actual state-derivative. However, in the transient phase, the derivative estimation errors can be large. Since CL relies on repeated use recorded data, large transient errors present a challenge in the development of a CL-based parameter estimator. If the derivative estimation errors at the points recorded in the history stack are large, then the corresponding errors in the parameter estimates will be large. Motivated by the results in [21], the aforementioned challenge is addressed in this paper by designing a novel purging algorithm to purge possibly erroneous data from the history stack. The closed-loop error system along with the purging algorithm is modeled as a switched nonlinear dynamical system. Provided enough data can be recorded to populate the history stack after each purge, the developed method ensures asymptotic convergence of the error states to the origin.
The PE condition can be shown to be sufficient to ensure that enough data can be recorded to populate the history stack after each purge. Since PE can be an impractical requirement in many applications, this paper examines the behavior of the switched error system under a relaxed finite excitation condition. Specifically, provided the system states are exciting over a sufficiently long finite time-interval, the error states decay to an ultimate bound.
Furthermore, the ultimate bound can be made arbitrarily small by increasing the learning gains. Simulation results are provided to demonstrate the effectiveness of the developed method under measurement noise.

II. SYSTEM DYNAMICS
The system dynamics are assumed to be nonlinear, uncertain, and control-affine 1 , described by the differential where the function f : R n×m → R n is locally Lipschitz. It is assumed that the dynamics can be split into a known component and an uncertain component with parametric uncertainties that are linear in the parameters. That is, where f 1 : R n → R n , g : R n → R n×m and Y : R n → R n×P are known and locally Lipschitz continuous. The constant parameter vector θ ∈ R P is unknown, with a known bound θ ∈ R >0 such that θ < θ. The objective is to design a parameter estimator to estimate the unknown parameters. The system input is assumed to be a stabilizing controller such that x,ẋ, u ∈ L ∞ . 1 The system state x is assumed to be available for feedback, and the state-derivativeẋ is assumed to be unknown.

III. CL-BASED ADAPTIVE DERIVATIVE ESTIMATION
Letx ∈ R n andẋ ∈ R n denote estimates of the state x, and the state-derivativeẋ, respectively. Letθ ∈ R P denote an estimate of the unknown vector θ. To achieve convergence of the estimateθ to the ideal parameter vector θ, a CL-based parameter estimator is designed. The motivation behind CL is to adjust the parameter estimates based on an estimate of the parameter estimation errorθ, defined asθ θ −θ, in addition to the state estimation errorx. Sinceθ is not directly measurable, the subsequent development exploits the fact that the term Y (x)θ can be computed as provided measurements of the state-derivativeẋ are available. In CL results such as [8], [17], [18], [23], it is assumed that the state-derivatives can be computed with sufficient accuracy at a past time instance by numerically differentiating the recorded data. An approximation of the parameters estimation errors is then computed as Y ( denotes the system state at a past time instance t j ,ẋ j denotes the numerically computed state-derivative at t j , and d is a constant of the order of the error betweenẋ j andẋ j . While the results in [19] establish that, provided d is bounded, the parameter estimation errorθ can be shown to decay to a ball around the origin, the focus is on the analysis of the effects of the differentiation error, and not on development of algorithms to reduce the parameter estimation error.
In this paper, a dynamically generated estimate of the state-derivative is used instead of numerical smoothing.
The parameter estimation error is computed at a past recorded data point as be a history stack containing recorded values of the state, the control, and the state-derivative estimate. Each tuple ẋ j , x j , u j is 1 The focus of this paper is adaptive estimation, and not control design. Even though most adaptive controllers are designed based on an estimate of the unknown parameters, parameter estimation can often be decoupled from control design. For example, the adaptive controller in [22] can guarantee x,ẋ, u ∈ L∞ for a wide class of adaptive update laws. Under the additional assumption thatu ∈ L∞, the developed technique can be extended to include linearly parameterized nonaffine systems, that is, referred to as a data-point in H. A history stack H is called "full rank" if the state vectors recorded in H satisfy Based on the subsequent Lyapunov-based stability analysis, the history stack is used to update the estimateθ using the following update law: wherex = x −x ∈ R n , Γ ∈ R P ×P and k ∈ R are positive and constant learning gains.
The update law in (2) drives the parameter estimation error to a ball around the origin, the size of which is of the order ofẋ j . Hence, to achieve a lower parameter estimation error, it is desirable to driveẋ to the origin. Based on the subsequent Lyapunov-based stability analysis, the following adaptive estimator is designed to generate the state-derivative estimates:ẋ where µ ∈ R n is an auxiliary signal and k 1 , α 1 ∈ R >0 and γ 1 ∈ [0, 1] are positive constant learning gains.

A. Purging of history stacks
The state-derivative estimator in (3) relies on feedback of the state estimation errorx. In general, feedback results in large transient estimation errors. Hence, the state-derivative estimation errorsẋ j associated with the tuples ẋ j , x j , u j recorded in the transient phase can be large. The results in [19] imply that the parameter estimation errors can be of the order of max j ẋ j . Hence, if a history stack containing data-points with large derivative estimation errors is used for CL, then the parameter estimates will converge but the resulting parameter estimation errors can be large. To mitigate the aforementioned problem, this paper introduces a new algorithm that purges the erroneous data in the history stack as soon as more data is available. Since the estimator in (3) results in exponential convergence ofẋ to a neighborhood ofẋ, newer data is guaranteed to represent the system better than older data, resulting in a lower steady-state parameter estimation error. The following section details the proposed algorithm.

B. Algorithm to record the history stack
The history stack H is initialized arbitrarily to be full rank. An arbitrary full rank initialization of H results in a σ−modification (cf. [24]) like adaptive update law that keeps the parameter estimation errors bounded. However, since the history stack may contain erroneous data, the parameters may not converge to their ideal values.
In the following, a novel algorithm is developed to keep the history stack current by purging the existing (and possibly erroneous) data and replacing it with current data. The data collected from the system is recorded in an auxiliary history stack G ẋG . The history stack G is initialized such that and is populated using a singular value maximization algorithm [17]. Once the history stack G becomes full rank with a minimum singular value that is above a (static or dynamic) threshold, H is replaced with G, and G is purged. 2 In this paper, a dynamic threshold is used which is set to be a fraction of the highest encountered minimum singular value corresponding to H up to the current time.
In the subsequent Algorithm 1, a piece-wise constant function δ : R ≥0 → R ≥0 , initialized to zero, stores the last time instance when H was updated and a piecewise constant function η : where σ min denotes the minimum singular value. The constant ξ ∈ (0, 1) denotes the threshold fraction used to purge the history stack, and T ∈ R is an adjustable positive constant.
The following analysis establishes that if the system states are persistently exciting (in a sense that will be made clear in Theorem 1) then the parameter estimation error asymptotically decays to zero. Furthermore, it is also established that if the system states are exciting over a finite period of time, then the parameter estimation error can be made as small as desired provided T and the learning gains are selected based on the sufficient conditions introduced in Theorem 2.

A. Asymptotic convergence with persistent excitation
Purging of the history stack H implies that the resulting closed-loop system is a switched system, where each subsystem corresponds to a history stack, and each purge indicates a switching event. 3 To facilitate the analysis, let ρ : R ≥0 → N denote a switching signal such that ρ (0) = 1, and ρ (t) = j + 1, where j denotes the number of times the update H ← G was carried out over the time interval (0, t). In the following, the subscript s ∈ N denotes the switching index, and H s denotes the history stack corresponding to the s th subsystem (i.e., the history stack active during the time interval {t | ρ (t) = s}), containing the elements ẋ sj , x sj , u sj M j=1 . To simplify the notation, Note that A s : R ≥0 → R P ×P and Q s : Using (2) and (4), the dynamics of the parameter estimation errorθ can be written aṡ To establish convergence of the state-derivative estimates, a filtered tracking error r ∈ R n is defined as r ẋ +α 1x .
Using (1), (3), and (5), the time derivative of the filtered tracking error can be expressed as, where To facilitate the stability analysis, let F , F 1 , x, Y , and Γ be constants such that for all t ∈ R ≥0 . The following stability analysis is split into three parts. Under the temporary assumptions that the error statesθ,x, and r are bounded at a switching instance and that the norms of the state-derivative estimates stored in the history stack are bounded, it is established in Part 1 that the error statesθ,x, and r decay to a bound before the next switching instance, where the bound depends on the derivative estimation errors. Under the temporary assumption that the error statesx and r are bounded at a switching instance, it is established in Part 2 that the derivative estimation errorẋ can be made arbitrarily small before the next switching instance by increasing the learning gains. In Part 3, the temporary assumptions in Parts 1 and 2 are relaxed through an inductive argument where the results from Part 1 and Part 2 are used to conclude asymptotic convergence of the error statesθ,x, and r to the origin.
Part 1: Boundedness of the error signals: Let Z r TxTθT T ∈ R 2n+L and let V : R 2n+P → R ≥0 denote a candidate Lyapunov function defined as Using the Raleigh-Ritz Theorem, the Lyapunov function V can be bounded as where v 1 2 max 1, λ max Γ −1 , v 1 2 min 1, λ min Γ −1 . The subsequent stability analysis assumes that the learning gains k, k 1 , and α 1 , and the matrices A s satisfy the following sufficient gain conditions: 4 The following Lemma establishes boundedness of the error state Z.
Lemma 1. Let T ∈ R >0 be a constant such that ρ (τ ) = s, for all τ ∈ [t, t + T ). Assume temporarily that there exist constants H s , V s ∈ R >0 such that the elements of H s satisfy ẋ j ≤ H s , for all j ∈ {1, · · · M }, and that the candidate Lyapunov function satisfies V (Z (t)) ≤ V s . Then, the candidate Lyapunov function V is bounded as s . Furthermore, the parameter estimation error can be bounded as , and Q s Q s .
Proof: Using (5)-(6), the time derivative of the candidate Lyapunov function V can be written aṡ Provided the sufficient conditions in (10) are satisfied, the Lyapunov derivative can be bounded aṡ Using the hypothesis that the elements of the history stack are bounded and the fact that u is stabilizing, the Lyapunov derivative can be bounded asV

Using the comparison lemma [25, Lemma 3.4],
Hence, using the definition of Z and the bounds in (9), the bound in (12) is obtained.
Part 2: Exponential decay ofẋ: Let Z r r TxT T ∈ R 2n and let V r : R 2n → R ≥0 be a candidate Lyapunov function defined as V r (Z r ) Z r 2 . The following lemma establishes exponential convergence of the derivative estimation error to a neighborhood of the origin using Lemma 1.

Lemma 2.
Let all the hypotheses of Lemma 1 be satisfied. Furthermore, assume temporarily that there exists a constant V rs such that the candidate Lyapunov function V r satisfies V r (Z r (t)) ≤ V rs . Then, the Lyapunov function V r is bounded as where ν r = min k1 2 , α 1 and ι rs (θsF +kY Γ(θsA+Q s )) 2

k1
. Furthermore, given a constant r ∈ R >0 , the gain k 1 can be selected large enough such that ẋ (t + T ) ≤ r .
Proof: Using (6), the time derivative of the candidate Lyapunov function V r can be written aṡ Completing the squares and using Lemma 1, the Lyapunov derivative can be bounded aṡ Using the comparison lemma, [25, Lemma 3.4] Using the fact that,ẋ = r − α 1x the state-derivative estimation error can be bounded as Based on (13), given V rs ≥ V r (Z r (t)), r > 0, the gain k 1 can be selected large enough so that V r (Z r (t + T )) ≤ 2 r (1+α1) . Hence, given V rs , r > 0, the gain k 1 can be selected to be large enough so that ẋ (t + T ) ≤ r . Part 3: Asymptotic convergence to the origin: Lemmas 1 and 2 employ the temporary hypothesis that the statederivative estimatesẋ j stored in the history stack remain bounded. However, since the estimatesẋ are generated dynamically using (3), they can not be guaranteed to be bounded a priori. In the following, the results of Lemmas 1 and 2 are used in an inductive argument to show that all the states of the dynamical system defined by (5)- (6) remain bounded and decay to the origin asymptotically provided enough data can be recorded to repopulate the history stack after each purge. 5 Theorem 1. Provided the history stacks H and G are populated using Algorithm 1, the learning gains are selected to satisfy V r1 > ιr1 vr and the sufficient gain conditions in (10), a bound V 1 ∈ R >0 is known such that V 1 > max v v ι 1 , V (Z (0)) , and provided the system states are exciting such that the history stack H can be persistently purged and replenished, i.e., then, θ (t) → 0, r (t) → 0, and x (t) → 0 as t → ∞.
Proof: Let {T s ∈ R ≥0 | s ∈ N} be a set of switching time instances defined as T s = That is, for a given switching index s, T s denotes the time instance when the (s + 1) th subsystem is switched on. To facilitate proof by mathematical induction, assume temporarily that the hypotheses of Lemmas 1 and 2 are satisfied for t ∈ [0, T s ) for some s.
Furthermore, assume temporarily that the following sufficient condition is satisfied: Then, using (11) and (13), the Lyapunov functions V and V r can be bounded as V (Z (t)) ≤ W (t)  (11) and (13), respectively, and W , and W r denote the envelopes that bound the Lyapunov functions V and V r , respectively. Using the bounds on the Lyapunov functions, the bounding envelopes at consecutive switching instances can be related as and using (15) and the hypotheses of Theorem 1, then W (T s ) < W (T s−1 ) and W r (T s ) < W r (T s−1 ).
Since the history stack H 1 is selected at random to include bounded elements, all the hypotheses of Lemmas 1 and 2 are satisfied over the time interval [0, Using the bounds in (7), Q 1 can be computed as vr stated in Theorem 1, it can be concluded that W (T 1 ) < W (0) and W r (T 1 ) < W r (0). Selecting V 2 = W (T 1 ) and V r2 = W r (T 1 ), it can be concluded that V 2 < V 1 and V r2 < V r1 .
Hence, by mathematical induction, the hypotheses of Lemmas 1 and 2 are satisfied for all t ∈ R ≥0 , and is common among all the subsystems, Z (t) → 0 as t → ∞ Theorem 1 implies that provided the system states are persistently excited such that the history stack H can always be replaced with a new full rank history stack, the state-derivative estimates, and the parameter estimate vector asymptotically converge to the state-derivative and the ideal parameter vector, respectively. However, from a practical perspective, it may be undesirable for a system to be in a persistently exciting state, or excitation beyond a certain finite time-interval may not be available. If excitation is available only over a finite time-interval, then the parameter estimation errors can be shown to be uniformly ultimately bounded, provided the history stacks are updated so that the time interval between two consecutive updates, i.e., the dwell time, is large enough. Algorithm 1 guarantees a minimum dwell time between two consecutive updates provided the constant T is selected large enough.

B. Ultimate boundedness under finite excitation
For notational brevity, let β 1 Theorem 2. Let all the hypotheses of Theorem 1 be satisfied, except for the persistent excitation hypothesis in (14).
Let T ∈ R >0 be a time instance such that T s < T for some s > 2. Provided the history stacks are updated and populated using Algorithm 1, where the minimum dwell time satisfies and the sufficient gain condition k 1 > 8vrv 2 M Y (1+α1)β1β 2 Proof: Since the history stack H 1 is selected at random, all the hypotheses of Lemmas 1 and 2 are satisfied Using the bounds in (7), Q 1 can be computed as Q 1 = qM Y F 1 + H 1 , where q > 1 is an adjustable parameter.
vr . Let 1 > 0 be a constant, to be selected later. Provided the gain k 1 is selected such that k 1 > 2vr(1+α1)β1 2 1 , then (1+α1) . Since T 2 < T by hypothesis, the Lyapunov envelope W r decays over the time interval [T 1 , T 2 ); hence, sup t∈[T1,T2) ẋ (t) ≤ 1 , which implies Q 3 can be selected as Q 3 = M Y 1 and ι 3 = β 2 M Y 1 . Selecting 2vβ2M Y , the inequality ι 3 ≤ vv 2v is obtained. If the history stack H is not updated after the time instance T s , then using an inductive argument similar to the proof of Theorem 1, it can be concluded that all the hypotheses of Lemma 1 are satisfied for all t ∈ [T s , ∞).

Hence, the Lyapunov function
Since Algorithm 1 is designed to allow a minimum dwell time of T seconds, T s ≥ sT and T s − T j ≥ (s − j) T . Hence,

VI. SIMULATION RESULTS
The developed technique is simulated using a model for a two-link robot manipulator arm. The four-dimensional state of the model is denoted by x The dynamics of the model are described by (1), where In (17),   Figure 1a shows the evolution of the system state, where the added noise signal can be observed. Figure 1b demonstrates convergence of the unknown parameters to a neighborhood of their true values, where the dashed lines represent the true values. Figure 2a shows the convergence of the state estimation error to a ball around the origin. Figure 2b shows the convergence of the state-derivative estimation error to a ball around the origin.
The transients in Figures 2a and 2b necessitate the need for history stack purging. Figure 3a shows the minimum singular value of the history stack H. The singular value is increasing because of the thresholding algorithm. In this simulation, the threshold parameter ξ is set to one. Figure 3b shows the increments of the purging index, It can be observed that the history stack gets purged faster initially as transients offer significant data, and then the rate of purging levels off approximately to a constant as the system achieves steady state. 6 VII. CONCLUDING REMARKS A concurrent learning-based parameter estimator is developed for a linearly parameterized control-affine nonlinear system. An adaptive observer is employed to generate the state-derivative estimates required for concurrent learning.
The developed technique is validated via simulations on a nonlinear system where the state measurements are corrupted by additive Gaussian noise. The simulation results indicate that the developed technique yields better results than numerical differentiation-based CL, even more so as the variance of the additive noise is increased.
Even though the simulation results indicate a degree of robustness to measurement noise, the theoretical development does not account for measurement noise.   Measurement noise affects the developed parameter estimator in two ways. An error is introduced in the statederivative estimates generated using the adaptive observer, and an error is introduced via the history stack since the state measurements recorded in the history stack are corrupted by noise. The former can be addressed if a noise rejecting observer such as a Kalman filter is used to generate the state-derivative estimates. The latter can be addressed by the use of an inherently noise-robust function approximation technique, e.g., a Gaussian process, to approximate the system dynamics. An extension of the developed parameter estimator that is provably robust to measurement noise is a topic for future research.