{'fraction_non_alphanumeric': 0.049022458027472925, 'fraction_numerical': 0.016062213823679047, 'mean_word_length': 4.300269645608629, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Deterministic and Stochastic Approaches to Supervisory Control Design for Networked Systems with Time-Varying Communication Delays

This paper proposes a supervisory control structure for networked systems with time-varying delays. The control structure, in which a supervisor triggers the most appropriate controller from a multi-controller unit, aims at improving the closed-loop performance relative to what can be obtained using a single robust controller. Our analysis considers average dwell-time switching and is based on a novel multiple Lyapunov-Krasovskii functional. We develop stability conditions that can be verified by semi-definite programming, and show that the associated state feedback synthesis problem also can be solved using convex optimization tools. Extensions of the analysis and synthesis procedures to the case when the evolution of the delay mode is described by a Markov chain are also developed. Simulations on small and large-scale networked control systems are used to illustrate the effectiveness of our approach.


Introduction
Networked control systems are distributed systems that use communication networks to exchange information between sensors, controllers and actuators [1,2]. The networked control system architecture promises advantages in terms of increased flexibility, reduced wiring and lower maintenance costs, and is finding its way into a wide range of applications, from automobiles and transportation to process control and power systems, see e.g., [2] - [5].
The use of a shared communication medium introduces time-varying delays and information losses which may deteriorate the system's performance, even to the point where the closed-loop system becomes unstable. A conservative approach is to design a robust controller that considers the worst-case delay. However, this might cause poor performance if the actual delay is only rarely close to its upper bound. Therefore, there is currently a renewed interest in adapting the control law to the delay evolution (e.g., [6] - [11]). Inspired by the communication delays that we have experienced in applications, see Figure 1, we design a supervisory control scheme in the sense of [12]. This control architecture consists of a finite number of controllers, each designed for a bounded delay variation (corresponding, e.g., to low, medium and high network load) and a supervisor which orchestrates the switching among them.
The analysis of switched systems with fixed time-delays is challenging and has attracted significant attention in the literature, e.g., [7,13,14,15]. Only recently, however, attempts to analyze switched systems with time-varying delays have begun to appear. Distinctively, [8] constructed multiple Lyapunov-Krasovskii functionals that guarantee closedloop stability under a minimum dwell-time condition for interval time-varying delays. An alternative approach to deal with time-varying delays is to assume that they evolve according to a Markov chain and develop conditions that ensure (mean-square) stability, see e.g., [16] - [21]. The work in [16] assumed that the time delay never exceeds a sampling interval, modeled its evolution as a Markov process, and derived the associated LQG-optimal controller. However, this formulation is not able to deal with longer time-delays. The work in [17] [24]. The delay exhibits distinct mode changes (here corresponding to one, two or three-hop communication) and varies around its piecewise constant modedependent mean. Similar behavior was reported by [25], who measured the delay of sensor data sent over a CAN bus. Their delay varied between 10-20 ms, but increased abruptly to around 150 ms under certain network conditions. linear system formulation, which allows longer (but bounded) time delays, and posed the design of a mode-dependent controller as a non-convex optimization problem. Complementary to these discrete-time formulations, [19] - [21] have investigated the mean-square stability of continuous-time linear systems with random time delays using stochastic Lyapunov-Krasovskii functionals. The papers [22,23] have applied these techniques to networked control systems with random communication delays and synthesized mode-dependent controllers.
In this paper, we analyze our proposed supervisory control structure by combining a novel multiple Lyapunov-Krasovskii functional with the assumption of average dwell-time switching. The average dwell-time concept, introduced in [26], is a natural deterministic abstraction of load changes in communication networks, where minimal or maximal durations for certain traffic conditions are hard to guarantee. We demonstrate that the existence of a multiple Lyapunov-Krasovskii functional that ensures closed-loop stability under average dwell-time switching can be verified by solving a set of linear matrix inequalities. In addition, we show that the state feedback synthesis problem for the proposed supervisory control structure can also be solved via semi-definite programming. A similar analysis for Markovian time-delays is developed based on a slightly less powerful Lyapunov-Krasovskii functional than the one underpinning our deterministic analysis. Also in this case, we manage to design mode-dependent state feedback controllers using convex optimization.
The organization of the paper is as follows. Section 2.1 presents the supervisory control structure and formalizes the relevant analysis and synthesis problems in a deterministic setting. In Section 2.2, multiple Lyapunov-Krasovskii functionals are constructed for establishing exponential stability of supervisory control systems under average dwelltime switchings. Additionally, LMI conditions that verify the existence of such a multiple Lyapunov functional are derived. State-feedback synthesis conditions are also given in Section 2.3. Section 3 develops a similar analysis framework for stochastic delays. Section 3.1 formulates switched control system problem introduced in Section 2 as a Markovian jump linear system. Section 3.2 develops stochastic exponential mean-square stability conditions for the supervisory control system under stochastic delays. The corresponding state-feedback synthesis conditions are proposed in Section 3.3. Numerical examples are used to demonstrate the effectiveness of the proposed techniques in Section 4. Finally, Section 5 concludes the paper.
Notation: Throughout this paper, R n denotes the n-dimensional Euclidean space, R m×n is the set of all m × n real matrices, and S n ++ denotes the cone of real symmetric positive definite matrices of dimension n. For a real square matrix M we define M S M + M where M is its transpose. Additionally, ' ' represents symmetric terms in symmetric matrices and in quadratic forms, ⊗ denotes the Kronecker product, and R ≥0 R >0 is the set of nonnegative (positive) real numbers. Lastly, col(λ i ) is the column vector with components λ i .

System Modeling
We consider the supervisory control system in Figure 2. Here, G is the plant to be controlled, described bẏ where A ∈ R n×n and B ∈ R n×m . The network is modelled as a time-varying delay τ σ(t) (t) where σ : R ≥0 → M with M = {1, . . . , M} is the mode (operating condition) of the network. We assume that the delay in each mode is bounded, The multi-controller unit uses the mode signal to select and apply the corresponding mode-dependent feedback law In this way, the closed-loop system is described by the following switched linear system with time-varying delay where A σ(t) BK σ(t) ∈ R n×n and ϕ(t) ∈ C [−h M+1 , 0], R n is the initial function belonging to C [−h M+1 , 0], R n , the Banach space of continuous functions defined on [−h M+1 , 0]. Definition 2.1. The system (3) is exponentially stable under the switching signal σ(t) if there exist positive constants γ and α such that the solution of x(t) of the system (3) satisfies In order to guarantee exponential stability, we will put restrictions on the switching signal σ(t). Specifically, we will assume that the signal satisfies an average dwell-time condition in the following sense. [27]). We denote the number of jumps of a switching signal σ on the interval (t, T ) by N σ (T, t). Then we say that σ has the average dwell-time τ a if there exist two positive numbers N 0 and τ a such that

Definition 2.2 (Liberzon
The set of all switching signals satisfying the above condition is denoted by S[τ a ]. We consider two specific problems in this section. The first is to verify that the switched linear system (3) is exponentially stable under average dwell-time switching. The second one is to design state feedback controllers for each mode such that the supervisory control system is exponentially stable with guaranteed convergence rate.

Exponential Stability Analysis Using Multiple Lyapunov -Krasovskii Functionals
The exponential stability of switched system (3) is equivalent to the existence of a scalar α ∈ R ≥0 such that e αt ||x(t)|| converges asymptotically to zero for each σ ∈ S[τ a ]. To characterize the rate of convergence of the system (3), let us consider the change of variables ξ(t) e αt x(t). Then we have: where Note that the asymptotic stability of (4) implies that the original system is exponentially stable with decay rate α. However, this change of variables introduces a time-varying coefficient in the switched system model (4). Similar to [28], we can exploit the (mode-dependent) bounds on the delay and rewrite (4) in a polytopic form. Specifically, we express the term e ατ i (t) as a convex combination of the bounds e αh i and e αh i+1 : where λ 1 (t), λ 2 (t) ∈ R ≥0 and λ 1 (t) + λ 2 (t) = 1, ∀t ∈ R ≥0 . The delayed differential equation (3) is then rewritten aṡ where A α (αI n + A) and We combine a novel multiple Lyapunov-Krasovskii functional with the dwell-time approach of [26] to establish exponential stability of the switched system (3). For ease of notation, we state the theorem for the case of two delay modes only, but the approach extends immediately to a system with M modes.
2} and a constant scalar µ > 1 satisfying P i ≤ µP j , Q ik ≤ µQ jk , R ik ≤ µR jk , S ik ≤ µS jk and T ik ≤ µT jk , ∀i, j, k ∈ {1, 2} such that the LMIs Our claim follows if we can find Lyapunov-Krasovskii functionals V i (t) that guarantee decay rate α while in mode i and a constant µ > 1 such that V i (t) ≤ µV j (t) ∀i, j ∈ {1, 2}. Then, by [26,Theorem 1], (3) is exponentially stable for every switching signal σ with average dwell-time τ a > τ * a = ln µ α . We consider the following Lyapunov-Krasovskii functional, inspired from [29], The derivative of V i (t) along the trajectory of the system is given bẏ Using Jensen's inequality [30], the integral term To upperbound the integral term δ kξ (s)T ikξ (s)ds, we use the reciprocally convex combination from [31]: Substituting (11) and (12) into (10), we compute an upper bound for the Lyapunov functional (9) aṡ for all i ∈ {1, 2}. Then, for two different modes, we form the following two matrices: where where By applying the Schur complement twice to Γ i j to form Γ i j , we arrive at the equivalent condition: As argued above, the condition is satisfied for all λ i (t) if Γ 1i and Γ i2 are both negative definite. By guaranteeing that Γ i (t) < 0, we ensure that the dynamics in each fixed mode is exponentially stable with decay rate α. However, to guarantee stability for the switched system under the average dwell-time assumption, we also need to guarantee that for some µ > 1. Noting that V i (t) is linear in P i , Q ik , R ik , S ik and T ik , (16) is implied by the following conditions: . This concludes the proof.
Proposition 2.4. A lower bound on the average dwell-time ensuring the global stability of switched delay system (3) is given by τ • a = ln µ/α • where α • is the optimal value of the convex optimization problem LMIs (6), (7), and (8), 6 Due to the presence of multiple product terms αP i and e αh i+ j−1 A i P i in (6) and (7), the problem cannot be solved directly using semidefinite programming. However, the problem is easily seen to be quasi-convex. Hence, we can solve it by bisection in α. Since the decay rate α is inversely proportional to τ a , this solution procedure gives us a lower bound on the allowable average dwell-time τ a . If we can guarantee that the average dwell-time between mode changes in the communication network is larger than this bound, then global stability of the closed-loop is guaranteed.

State-Feedback Controller Design
In this section, we will extend our analysis conditions to mode-dependent state feedback synthesis for the supervisory control structure introduced in Section 2.1. More precisely, we consider a linear time-invariant planṫ where the control input is a mode-dependent linear feedback of the delayed state vector, i.e, The design problem is to find feedback gain matrices K i that ensure closed-loop stability for all switching signals in S[τ a ]. Clearly, this problem is closely related to the stability analysis problem considered in Section 3, since the supervisory control structure induces a switched linear system on the form (3) with A σ(t) = BK σ(t) . We have the following result: Theorem 2.5. For a given decay rate α > 0, there exists a state-feedback control of the form (18) which exponentially Proof: The structure of (6) and (7) is not suitable for the synthesis of a state-feedback controller due to the presence of multiple product terms A α S ik , A α T ik , A α i j S ik and A α i j T ik . These product terms prevent finding a linearizing change of variable even after congruence transformation. Instead, we will use the relaxation term introduced in Briat et. al [32] to decouple the products at the expense of an increased conservatism. Denote (19) and (20) by Θ 1 j and Θ 2 j , respectively. Then we prove that Θ i j < 0 ∀i ∈ {1, 2} implies the feasibility of (6) and (7). Note that Θ i j can be decomposed as where U 1 = − I n A α A α 1 j 0 n×4n I n 0 n×4n , V 1 = I n 0 n×11n , U 2 = − I n A α 0 n A α 2 j 0 n×3n I n 0 n×4n and V 2 = I n 0 n×11n . Then invoking the projection lemma [33], the feasibility of Θ i j < 0 implies the feasibility of the LMIs where N U i and N V i are basis of the null space of U i and V i , respectively. After some tedious calculations, we can show that LMIs (24) and (25) are equivalent to (6) and (7) showing that Θ i j < 0 ∀i ∈ {1, 2} implies the feasibility of (6) and (7). Moreover, LMI (23) characterizes the conservatism of the relaxation.
Since LMIs (24) and (25) do not include any multiple product, it can easily be used for controller design. Hence, it is possible to use congruence transformations and change of variables so as to design the state-feedback controller. Performing a congruence transformation with respect to matrix I 12n ⊗ X −1 and applying the following linearizing (19) and (20).

Stochastic Switched Systems
Our deterministic modeling framework has several advantages: it allows to model long time-delays, is able to account for mode-dependent delay bounds and admits a convex formulation of the (mode-dependent) state feedback synthesis problem. However, it also has a disadvantage in that it does not allow to account for more detailed knowledge about the evolution of the delay mode beyond the average dwell-time. It is therefore interesting to derive similar results when the delay mode varies according to a Markov chain, cf. [19,20,21,34]. Such results will be developed next.

System Model
Let us consider a dynamical system in a probability space (Ω, F , P), where Ω is the sample space, F is the σalgebra of subsets of the sample space and P is the probability measure on F . Over this probability space, we consider the following class of linear stochastic systems with Markovian jump parameters and mode-dependent time delays Here, x(t) ∈ R n is the state, A ∈ R n×n and A r(t) ∈ R n×n are the known system matrices while r t , t ∈ R ≥0 is a homogeneous, finite-state Markovian process with right continuous trajectories and taking values in the finite set M = {1, · · · , M}. The Markov process describes the switching between the different modes and its evolution is governed by the following transition probabilities where π i j is the transition rate from mode i to j with π i j ≥ 0 when i j and π ii = − N j=1, j i π i j and o(∆) is such that lim ∆→0 o(∆) ∆ = 0. Furthermore, τ r(t) (t) is the time-varying stochastic delay function satisfying

Finally, ϕ(t) is a vector-valued initial continuous function defined on the interval [−h
holds for any finite ϕ(t 0 ) ∈ R n defined on [−h M+1 , 0] and any initial mode r t 0 ∈ M.

Exponential Stability Analysis Using Stochastic Lyapunov-Krasovskii Functionals
In this subsection, we analyze the exponential stability of the Markovian jump linear system (26) using a similar approach to what we developed for the switched delay case. To portray the convergence rate of the system (26), we thus consider the change of variables ξ(t) e αt x(t) and finḋ where τ r(t) ∈ h r(t) , h r(t) . For each r(t) = i, ∀i ∈ M, we rewrite (27) aṡ Using the same polytopic approach as in Section 2, we express e ατ i (t) as a convex combination of its mode-dependent bounds: where λ 1 (t), λ 2 (t) ∈ R ≥0 and λ 1 (t) + λ 2 (t) = 1, ∀t ∈ R ≥0 . Thus, the stochastic switched system (26) can be defined, for each r(t) = i, ∀i ∈ M, asξ where A α (αI n + A) and Theorem 3.2. The Markovian jump linear system (26) is exponentially mean-square stable with a given decay rate α > 0 for randomly varying delays , 2}, S , T, Q, R ∈ S n ++ and Z ∈ R n×n such that the following LMIs hold for i, j ∈ {1, 2} Proof. We define a stochastic Lyapunov-Krasovskii functional V : R n × M → R ≥0 candidate for the system (26) as where Definition 3.3 (Øksendal [35]). The weak infinitesimal operator A of the Markovian process x(t), r t , t ≥ 0 is defined by Straightforward but tedious calculations yield that, for each r t = i, i ∈ M, along solutions of (26), we have δ iξ (s)Tξ(s)ds that appear in the preceding equalities as follows: where T Z T ≥ 0 holds. In addition, for the stochastic formulation, we need to upper bound a number of additional integrals. We do so by noting π i j ≥ 0 for j i and π ii ≤ 0, and that Now, substituting (38) -(42) into (33) -(37), we get the following inequality where ψ(t) = col ξ(t), ξ(t − τ i (t)), ξ(t − h i ), ξ(t − h i+1 ) . Note that AV(ξ t , r t ) is bounded by a quadratic function in ψ(t): where for all i ∈ {1, 2} and where φ i j A A α i j 0 n×2n for all i, j ∈ {1, 2}. As in Section 2, applying the Schur complement lemma to Γ i j to form Γ i j , we arrive at the equivalent condition This condition is satisfied for all λ i if Γ i1 and Γ i2 are both negative definite. Hence, satisfying the condition Γ i j < 0, we guarantee that the dynamics is exponentially stable with decay rate α. This concludes the proof.

State-Feedback Controller Design
Hereafter we concentrate our interest on extending analysis conditions to state-feedback synthesis for exponential mean-square stability of the Markovian jump linear system scheme introduced in Section 3.1. To this end, we consider a linear time-invariant plantẋ (t) = Ax(t) + Bu(t) where x(t) ∈ R n is the state and u(t) ∈ R m is the control input being mode-dependent linear feedback of the delayed state with the following control law when r(t) = i (and hence, τ i (t) ∈ [h i , h i+1 )), i ∈ M. Indeed, the design problem is to determine a set of statefeedback gain matrices K i that guarantees the closed-loop control stability in exponentially mean-squared sense for the transition rates Π = [π i j ] i, j=1,··· ,M . This problem is closely related to the stability analysis problem discussed in Section 3.2 because the control system structure can also be represented as a Markovian jump linear system on the form of (26) with A r(t) = BK r(t) . We have the following result.
Theorem 3.4. For a given decay rate α > 0, there exists a state-feedback control of the form (49) that stabilizes system (26) for randomly varying delays τ i ∈ [h i , h i+1 ), ∀i ∈ {1, 2} in exponentially mean-squared sense if there exist real constant matricesP i ,Q i ,R i ∈ S n ++ , ∀i ∈ {1, 2},S ,T ,Q,R ∈ S n ++ andX,Z ∈ R n×n such that the following LMIs hold for i, j ∈ {1, 2} and Proof. The proof of Theorem 3.4 is similar to that of Theorem 2.5 but an outline of the proof is included for completeness. Again, the structure of (31) is not adapted to the controller design due to the existance of the multiple product terms A α S , A α T , A α i j S and A α i j T preventing to find a linearizing change of variable even after congruence transformations. As a result, a relaxation approach is applied (as in § 2.3) to remove the multiple product terms preventing the change of variables. We let (51) be called Ψ i j and we prove the condition Ψ i j < 0. Similarly to Θ i j (but with the difference that we have X instead of X i ), Ψ i j can be decomposed as follows: where U i = − I n A α A α i j 0 n×2n I n 0 n×2n and V i = I n 0 n×7n . Then invoking the projection lemma [33], the feasibility of Ψ i j < 0 implies the feasibility of the LMIs where N U i and N V i are basis of the null space of U i and V i , respectively. Subsequently, the inequality (52) is obtained as We substitute the closed-loop system (48) into the inequality (31), and introduce a constant matrix X ∈ R n×n . Then, we compel X to be constant, and we perform a congruence transformation with respect to matrix I 8n ⊗ X −1 and apply the following linearizing change of variablesX To design a set of stabilizing controllers for Markovian jump system (as discussed in § 3.3) with M modes, 2M LMIs (e.g., (51)) whose dimensions are 8n × 8n, and also 2M + 1 small size LMIs (e.g., (50)) must be checked. All these LMIs include 3M + 6 matrix variables with n(n + 1)/2 decision variables.

Numerical Examples
We are now ready to demonstrate the proposed technique on numerical examples. Our first example, a simple DCmotor model, is included to demonstrate the flexibility of our control structure compared to a single robust controller. The second example is taken from wide-area power systems, and demonstrates that the numerical techniques scale to non-trivial system dimensions. Finally, we return to the small-scale example to illustrate the analysis procedures for the random Markovian delay model.

Small-scale Example: DC Motor
Consider the following linear systemẋ with a time-varying communication delay between sensor and controller that behaves as shown in Figure 3. The supervisor generates the switching signals shown in Figure 3 and triggers the most appropriate controller according to By solving the optimization problem (17)  Using the same class of Lyapunov-Krasovskii functionals, we design a classical state-feedback controller to compare the non-switching and switching control performance. Specifically, we use the Lyapunov-Krasovskii functional (9) with Q k = 0, R k = 0, S k = 0, T k = 0, ∀k > 1 and τ(t) ∈ [20, 300) ms, and find α = 1.72 for the state feedback gain  We observe that the switching controller has a better performance than the non-switching one, α • = 2.78 > α = 1.72, and that the controller in the low-delay mode can be made much more agressive when we use a mode-dependent controller. The improved convergence rate is confirmed by the simulations shown in Figure 3.
For different values of h 2 and h 3 , which define the boundary between the low-, medium-and high-delay mode, we compute the exponential decay rate α of the supervisory controller, and then try to find the largest h max for which a single mode-independent controller can guarantee the same decay rate. The results are depicted in Table 1. As can be seen, increasing h 2 and decreasing h 3 allow to guarantee an improved decay rate of the switching controller. To guarantee the same decay rate under a mode-indenendent controller, the maximum delay must be reduced, and sometimes significantly so.

Large Scale Example: Wide-Area Power Networks
To demonstrate the applicability of our methods to systems of higher dimension, we consider the IEEE nine-bus system [36] shown in Figure 4. We adopt a second order (swing) model with phase and frequency (δ i , ω i ) for all generators and use the Power System Analysis Toolbox [37] to obtain the following numerical model d dt  We assume that the phase and frequency of each bus can be measured and be communicated to a central controller. In wide-area power systems, the communication delays vary depending on communication technologies, protocols and network load. In this example, we assume that the delay varies between 20 and 110 ms (see, Figure 5) and that mode changes are such that the average dwell-time is guaranteed to be at least 0.35 seconds.
To begin, the supervisory controller is designed for the delay intervals [20,50) and [50,110) ms. We design supervisory control gains targeting a decay rate of α = 0.9, and use µ = 1.4 to guarantee τ a = 0.3739. As a result, we compute the following control gain matrices Solving the LMIs for this larger system takes a few hours, but appears to be numerically stable.
It turns out that an exponential decay rate of α = 0.9 can also be achieved by the mode-independent feedback gain which is less aggressive than the controller used in the low-delay mode of the switching controller. As depicted in Figure 5, simulations are carried out for the delay trace and corresponding switching singnals shown in the left of the same figure, the switching controller damps oscillations between generators G 2 and G 3 better than its modeindependent counterpart.

Markovian Jump Linear System Formulation
Finally, we return to the DC-motor example to illustrate the applicability of the analysis tools developed in Section 3 for random delays. We consider the system (53) with low traffic delays (i.e., τ L ∈ [20, 70)ms) and high traffic delays (i.e., τ H ∈ [70, 300)ms). The switching between these two modes is described by the following transition probability rate matrix: whose invariant distribution is π ∞ 1 = q/(p + q) and π ∞ 2 = p/(p + q). We vary the transition probabilities p and q, compute mode-dependent controllers and the rate of exponential mean-square stability. The results are summerized in Table 2. While mitigating max p, q , the decay rate is increasing and controllers are getting more aggressive. Furthermore, when π ∞ 1 decreases, we also observe a slight decrease in the decay rate. 17

Conclusion
This paper has been dedicated to supervisory control of networked control systems with time-varying delays. The main contribution of this paper is to develop a stability analysis and a state feedback synthesis technique for a supervisory control system that switches among a multi-controller unit based on the current network state. A novel class of Lyapunov-Krasovskii functionals were introduced that, somewhat remarkably, admit both the analysis and state feedback synthesis problems to be solved via convex optimization over linear matrix inequalities. In addition, we also investigated the corresponding problem for a class of stochastic systems with interval-bounded time-varying delay. Sufficient conditions were established without ignoring any terms in the weak infinitesimal operator of the Lyapunov Krasovskii functional by considering the relationship among the time-varying delay, its upper bound, and their difference. Finally, examples were given to show the effectiveness of the proposed analysis and synthesis techniques.