{'fraction_non_alphanumeric': 0.05530543333674409, 'fraction_numerical': 0.018469252020873836, 'mean_word_length': 4.303038524145415, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 18, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A family of extremum seeking laws for a unicycle model with a moving target: theoretical and experimental studies

In this paper, we propose and practically evaluate a class of gradient-free control functions ensuring the motion of a unicycle-type system towards the extremum point of a time-varying cost function. We prove that the unicycle is able to track the extremum point, and illustrate our results by numerical simulations and experiments that show that the proposed control functions exhibit an improved tracking performance in comparison to standard extremum seeking laws based on Lie bracket approximations.


Introduction
Extremum seeking typically refers to the problem of constructing a gradient-free control law that ensures the motion of a dynamical system to the minimum (or maximum) of a partially or completely unknown and possibly time-varying cost or performance function. Over the past decades, significant advances in the theory and applications of extremum seeking have been made, see, e.g., [15,29]. Today, there exist many ways to design and analyze extremum seeking laws exploiting, e.g., averaging and singular perturbation techniques, Lie bracket approximation techniques, least squares estimation approaches, stochastic and hybrid approaches, see, e.g., [1,5,6,11,12,14,16,17,21,22]. Many extremum seeking schemes use control functions depending on the current value of the cost function modulated by time-periodic oscillating excitation (or dither, learning) signals in order to explore and extract sufficient information from the dynamical system and/or from the unknown cost function to solve the extremum seeking problem.
The choice of the control function as well as the excitation signals plays an important role for the performance of the extremum seeking scheme [2,20,27,30]. In the recent paper [10], a broad family of control functions for extremum seeking schemes based on Lie bracket approximations was presented for systems with single-integrator dynamics and for time-invariant cost functions. This class of controls has several favorable properties including the possibility of adapting and constraining the amplitude of the excitation signal. Moreover, in the paper [7] the extremum seeking problem for time-varying cost functions has been considered in the framework of Lie bracket approximations, but again with single-integrator dynamics and with standard control functions as used in [6]. The first contribution of this paper is a whole family of control functions which enables a system with unicycle-type dynamics to approximate the gradient-like flow of a time-varying cost functions. This result justify the use of gradientfree controllers presented in [10] in time-varying extremum seeking problems. For the sake of simplicity, we consider a distance-like time-varying cost function. Such problems arise, for example, when a robot has to follow a moving target (tracking problem) and only the distance (but not the relative position) to the moving target can be measured. Although gradient-free control laws for the tracking of a moving target have been previously considered (see, e.g., [3,4,13,18,19,23,24,25,31,32,33]), the main advantage of the proposed family of extremum seeking laws is, on the one hand, the high flexibility in designing the control functions such that they meet further specifications like input constraints, and, on the other hand, the family of control functions ensures rigorous stability and tracking properties.
As it will be shown, some important control strategies in the proposed class are not continuously differentiable. In view of this, the second contribution of this paper is the relaxation of the "C 2 -requirement" for the Lie bracket approximation approach [6,7]. Instead, we will require the continuity of Lie derivatives. This result will allow us to exploit a much wider class of admissible extremum seeking laws. Extremum seeking systems with non-C 2 vector fields were also considered in [10,26] for time-invariant cost functions. However, the results of the above papers are not directly applicable for time-varying extremum seeking problems.
As the third contribution of this paper, we show by numerical simulations and experiments with a mobile robot that the high flexibility of the proposed control functions can be utilized to significantly improve the tracking behavior in comparison to standard extremum seeking approaches considered, for example, in [6].
The rest of this paper is organized as follows. In Section 2, we formulate the extremum seeking problem and recall some results on the Lie bracket approximation approach. Section 3 presents a class of extremum seeking laws for a unicycle-type system and stability results for time-varying cost functions. The numerical simulations and experiments for several extremum seeking laws with different qualitative properties are discussed in Section 4.

Problem statement
Consider a unicycle modelẋ 1 = u cos(Ωt), where x = (x 1 , x 2 ) T ∈ R 2 is the state, u ∈ R is the control, and Ω > 0 is constant angular velocity. These equations correspond to the standard unicycle model where the equation for the angular velocityθ = Ω, θ(0) = 0, has been eliminated (see, e.g., [25,8] for details). Note that the model with non-constant angular velocity can also be considered using, e.g., singular perturbations techniques [5]. In the sequel, R + denotes the set of all non-negative real numbers.
then the above extremum seeking problem leads to steering the control system (1) to an arbitrary small neighborhood of the curve x * (t) = γ(t).

Notations
For f i , f j : R × R n → R n , x ∈ R n , we denote the Lie derivative ; for a function h ∈ C 1 (R n ×R k ; R) and ξ ∈ R n , the gradient of h with respect to ξ is denoted as ∇ ξ h(ξ, η) = ∂h(ξ,η) ∂ξ T ; for a one-parameter family of non-empty sets L t ⊂ R n , t ∈ R + , B δ (L t )= ∪ y∈Lt {x∈R n : x−y <δ} is a δ-neighborhood of the set L t at time t, and B δ (L λ,t ) denotes the closure of B δ (L λ,t ). For m, n ∈ Z, the notation i = m, n means that i takes all possible values from the set {m, m + 1, . . . , n}.
(A3) the functions u j are Lipschitz continuous and T -periodic with some T >0, and Since in time-varying extremum seeking problems it is often necessary to investigate the stability of a family of sets (instead of a single set), we will make use of the following definitions which can be found, e.g., in [7].

Definition 1.
A family of non-empty sets L t ⊆ D, t ∈ R + , is said to be locally practically uniformly asymptotically stable for (3) if it is • practically uniformly stable: for any ε>0 there exist δ, ω 0 >0 such that, for all t 0 ∈R + and ω>ω 0 , the following property holds for the solutions of (3): •δ-practically uniformly attractive with someδ > 0: for every ε > 0 there exist t 1 ≥ 0 and ω 0 > 0 such that, for all t 0 ∈ R + and ω > ω 0 , the following property holds for the solutions of (3): • the solutions of (3) are practically uniformly bounded: for each δ>0 there are ε>0 and ω 0 >0 such that, for all t 0 ∈R + and ω>ω 0 , the following property holds for the solutions of (3): If the attractivity property holds for everyδ>0, then the family of sets L t is called semi-globally practically uniformly asymptotically stable for (3). For systems independent of ω we omit the terms practically and semi.
The following result from [7] allows to establish practical asymptotic stability properties of (3) from asymptotic stability properties of the so-called Lie bracket system. Theorem 1. Let (A1)-(A3) hold. Suppose that a family of sets L t ⊂ D is locally (globally) uniformly asymptotically stable for the Lie bracket systeṁ and suppose that there exists a compact set S⊂R n such that L t ⊆S for all t ∈ R + . Then L t is locally (semi-globally) practically uniformly asymptotically stable for (3).
In order to characterize the stability and tracking behavior of (1) with respect to time-varying cost functions of the form (2), we will consider the following family of level sets of the cost functionĴ: Let us emphasize that most of the results presented in the following apply to general timevarying cost functions not necessarily restricted to the form (2). However, in target tracking applications, the cost function often takes the form (2) with a possibly unknown γ.

Family of extremum seeking controls
In [10], we have introduced a novel family of extremum seeking controls for systems with integrator dynamics and time-invariant cost functions. In this section, we show that a similar result can be obtained for system (1) with time-varying cost functions.

Theorem 2. Consider the control function
where Then the Lie bracket system corresponding to the closed-loop system (1) with control function (5) has the formẋ

Remark 1.
In formula (6), we assume that F 1 (z) = 0 except for at most a countable set of isolated zeros Z * = {z * k }. We treat the function Ψ 1 (z) := dz F 1 (z) 2 as an antiderivative of defined on the open set R \ Z * , so that (6) holds as an identity with continuous functions in a neighborhood of each point z / ∈ Z * . As the functions F 1 and F 2 are assumed to be globally continuous, formula (6) is treated in the sense that Proof of Theorem 2: Substituting the controls (5) into system (1), we obtain a system of the form (3) given byẋ where v j are the new 2πk ω -periodic inputs, Direct construction of system (4) with the use of (6) completes the proof. Notice that the above control laws leave a lot of freedom for tuning by choosing the functions F 1 and F 2 appropriately.

Stability conditions
If the functions F 1 and F 2 are of class C 2 , then the stability properties of the unicycle model (1) controlled by (5) can be deduced from the stability properties of the corresponding Lie bracket system (7). This directly follows from Theorem 1: Corollary 1. Let the functions F s (Ĵ(·, ·)) satisfy (A1)-(A2), and suppose that a one-parameter family of sets L λ,t is locally (globally) uniformly asymptotically stable for (7) with some λ > 0, and there exists a compact set S⊂R n such that L λ,t ⊆S for all t ∈ R + , then L λ,t is locally (semi-globally) practically uniformly asymptotically stable for (1) with controls (5).
In combination with asymptotic stability conditions of families of sets for the Lie bracket system, the above result describes a solution to Problem 1 with a wide class of time-varying cost functionsĴ, provided that λ is small enough. Although, unlike [7], the Lie bracket system for (1) is not the exact gradient flow of the cost function, it admits the same asymptotic stability conditions for L λ,t as proposed in [7] because of the property (Φ(x, γ), ∇Ĵ(x, γ)) ≡ 0. However, many functions F 1 , F 2 described by (6) fail to satisfy the C 2 -condition at the origin (so that Corollary 1 and the results of [7] are not applicable), but exhibit much better performance in comparison with systems with smooth vector fields (see [26,10,28] and Section IV for some examples). To overcome such limitation, we will present stability results under relaxed assumptions. Note that, although extremum seeking problems for systems with non-C 2 vector fields were previously considered, e.g., in [26,10], the results of the above papers are not applicable because of several reasons. First, it is easy to see that the time-varying function x * (γ(t)) is not a solution of system (7), therefore, the considered problem cannot be reduced to control design in a neighborhood of an admissible trajectory. Second, the approximation result in [26] has been proved under the assumption that the Lie bracket system possesses C 2 -vector fields: . However, the function ϕ 1 , ϕ 2 in Theorem 2 do not necessary satisfy this requirement. The following result establishes the stability of the unicycle model (1) controlled by (5) under relaxed assumptions. For clarity of presentation and because of space limitations, our next theorem and its proof will be stated for system (1) with time-varying cost functions of the form (2). It is expected that similar results can be obtained for a wide class of time-varying cost functions (but with more involved conditions). We leave the general case for future studies. Theorem 3. Let the cost J = J(x − γ(t)) be of the form (2), ρ > 0, D = ∪ t≥0 L ρ,t , and F 1 , F 2 be defined from (6). Assume that: (B2) the first-order partial derivatives of F i • J and of L F i (F j • J) are uniformly bounded in D \ {0} for all i, j, k ∈ {1, 2}; (B3) γ ∈ C 1 (R + ; R 2 ), and there exists a ν > 0 such that γ(t) ≤ ν for all t ∈ R + .
The proof is in Appendix A. Note that the proof technique is similar to [10, Theorem 3]. However, since the results of [10] are proved for the case of time-invariant vector fields f i (x) and constant x * , they are not directly applicable. The proof of Theorem 3 requires some extensions of the approach of [10] to control-affine systems with time-varying vector fields and non-vanishing drift term. Furthermore, unlike many other results on the time-varying extremum seeking problems, we do not assume that γ(t) is uniformly bounded.

Numerical Simulations and Experiments
In this section, we illustrate our results with examples and discuss some interesting choices of the functions F 1 and F 2 in the control law (6).

Moving target tracking
Let Ω = 5, ω = 50, and γ(t) = (0.1t, sin(0.1t)) T , so that the cost function is of the form In all simulations, we assume that system (1) is initialized at x 1 (0) = −1, x 2 (0) = 1, the functions F 1 , F 2 satisfy (6), and ϑ = 1. For the first case, take Here and in the sequel, we denote u(t) := u ω (t, J(x − γ)). Such type of controls were introduced in [6] and also used in other classical extremum seeking approaches (e.g., [16]), possibly with additional filters. The main advantages of this control are its simple analytical form and applicability for a wide class of cost functions. The corresponding plots are shown in Fig. 1 (left). The following control, introduced in [25], possesses similar properties (and, moreover, has an a priori known bound): |u(t)| ≤ √ αω for any J and all x ∈ R 2 , t ≥ 0.
The corresponding plots are shown in Fig. 1 (center). Although the controls (11) and (12) possess several useful properties, they always lead to non-vanishing oscillations of the trajectories of the extremum seeking system. This can be explained, in particular, by the fact that the controls (11) and (12) do not vanish for J = 0, i.e., when approaching the target. Requiring F 1 (J), F 2 (J) → 0 as J → 0, it is possible to construct control laws which reduce the amplitude of oscillations and ensure better convergence properties. In particular, these properties are ensured with the following control proposed in [28]: In order to combine the advantages of controls (13) (i.e., vanishing amplitudes when reaching the target) and (12) (i.e., bounded excitation signals independent of the cost function), the following control function has been proposed in [10]: where φ = 1−e −|J(x−γ)| 1+e J(x−γ) , ψ = e |J(x−γ)| +2 ln(e |J(x−γ)| −1) for J =0, and u=0 for J=0. Fig. 1 (right) presents the simulation results for system (1) with the controls (14). It can be seen that the controls (13) and (14) exhibit smaller tracking error and the control amplitudes. However, it has to be noticed that both controls (13) and (14) exhibit better behavior of an extremum seeking system only in case of known minimal value of the cost function, and control (14) requires that x(t 0 ) is close enough to x * (γ(t 0 )) (under a proper scaling of the cost function) for better convergence properties.

Experimental results
The above examples show that the proposed new extremum seeking control laws perform very well in numerical simulations. In this section we want to illustrate that the benefits also transfer to an experimental setup. We validated the control on a three-wheeled mobile robot (see Fig. 2) both in a fixed and a moving target tracking scenario.
Due to limitations in the experimental setup we do not directly measure the distance to the target but evaluate it using (x 1 , x 2 )-position measurements of both the robot and the target obtained from tracking them with a camera.
In the fixed target scenario, we let ω = 3, Ω = 1.5 and assume the cost function to be of the form (2) with γ ≡ x = (0.5, 0.7) being the constant position of the target. We compared the control laws (11), (12) and (14) where the parameters α and κ were tuned under the assumption that the input is bounded as |u(t)| ≤ 0.4, see Table 1.

Control
Fixed target Moving target law α κ α κ  The experimental results are depicted in Fig. 3. Control law (11) shows the worst performance and does not converge very close to the target, even in much longer time. The performance of (12) and (14) is comparable in terms of the accumulated squared distance error and the convergence time. However, while control law (12) is non-vanishing and thus the robot circulates around the target in the end, the robot only makes small movements in the end when using control law (14), and the resulting total control effort is drastically reduced in   comparison to control law (12). The reason why the control input does not vanish completely is the imperfect rotational motion of the robot when u(t) = 0.
We compared the control laws (12) and (14). Again, the parameters α and κ were tuned under the assumption that the input is bounded as |u(t)| ≤ 0.4, see Table 1. The experimental results are depicted in Fig. 4 (left) for control (12) and in Fig. 4 (right) for control law (14). Both control laws achieve tracking the moving target, where control law (14) shows a better behavior in terms of the tracking error while requiring only approximately half the control effort.
All in all, the experimental results show that the new extremum seeking control laws can lead to improved performance also in practical implementations. Nevertheless, due to low upper limits for ω and Ω, there is still quite a gap between experimental and simulative results.

Conclusions
In this paper, a novel family of extremum seeking laws have been introduced for unicycle models. We have proved that the proposed controls can be utilized for tracking an extremum point of a time-varying cost function by extending the theoretical results from [10] and [7]. In particular, we have discussed how the results can be applied to moving target tracking problems. We have illustrated by simulations as well as experiments that the proposed family of extremum seeking laws performs remarkably well for these type of problems. Our next goals are to construct further extensions of the family of control functions (6) for more general classes of cost functions and to evaluate their performance with simulations and experiments.
Step 2. Representation of the solutions. Let us expand the solutions of system (17) into the Volterra-type series. From (P4) and (B1), the representation g j (J(ξ(t))) = g j (J(ξ 0 )) + t t 0 dg j (J(ξ(s))) ds ds Applying the same procedure to L F j (F i • J)(ξ(s)) and using we get where r(t) is the remainder, It can be shown that Recall that the above represenation of the solutuons of (17) is well-defined for all t ∈ [t 0 ; t 0 + T ] because of (P4) and (B1).
So, we get the contradiction which proves that there exists an N > 0 such that J(ξ(N T )) < δ 0 . Thus, we have two possibilities. If J(ξ(t)) < δ 0 < λ for all t ≥ N T , then the proof of the attractivity is completed. Otherwise, we recall from (P3) that J(ξ(t)) < λ for all t ∈ [N T, (N + 1)T ].