{'fraction_non_alphanumeric': 0.050607690775054795, 'fraction_numerical': 0.012592149830643554, 'mean_word_length': 4.429684119428819, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Anytime Control Under Practical Communication Models

In this article, we investigate a novel anytime control algorithm for wireless networked control with random dropouts. The controller computes sequences of tentative future control commands using time-varying (Markovian) computational resources. The sensor–controller and controller–actuator channel states are spatial- and time-correlated, respectively, and are modeled as a multistate Markov process. To compensate the effect of packet dropouts, a dual-buffer mechanism is proposed. We develop a novel cycle-cost-based approach to obtain the stability conditions on the nonlinear plant, controller, network, and computational resources.


I. INTRODUCTION
During the past few decades, significant attention has focused on embedded or networked control systems that have limited and timevarying controller's computation capability due to high requirements on multitasking operations. In particular, assuming constant and limited computational resources, bounds on computational time of specific optimization algorithms for achieving stability were derived in [1] and [2]. For time-varying computational resources, a dynamic computation task scheduling method was proposed for model-predictive controllers [3]. On-demand computation scheduling of control input based on plant states was investigated for periodic, event-triggered, and self-triggered policies in [4]- [6], respectively.
Another stream of research considers anytime algorithms for robust control and making efficient use of time-varying computational resources. In general, an anytime algorithm can provide a solution even with limited computational resources and refines the solution when more resources are available. In the pioneering work [7], an anytime control system was proposed, where the number of updated states varies with the available computation time known to the controller a priori. In [8], an anytime control algorithm for a multi-input linear system was proposed for the scenario when the computation availability is unknown a priori. The main idea was to first calculate the most important component of the control vector and then calculate the less important ones as more computational resource becomes available. In [9], a sequence-based anytime control method was proposed, which can calculate a tentative sequence of future control input for as many time steps as allowed by the available computational resources at each time step. The precalculated control sequence can compensate for the time steps when no computational resource is available for control calculations. Following this article, sequence-based anytime control systems with Markovian processor availability, event-triggered sensor updates, and multiple control laws were investigated in [10]- [12], respectively. In [7]- [10], the sensor-controller (S-C) and controlleractuator (C-A) channels were assumed to be perfect and error-free.
In [11] and [12], the S-C channel was assumed to have independent and identically distributed (i.i.d.) packet dropouts, 1 i.e., only binary-level (ON-OFF) channel states were considered, while the controller-sensor channel was assumed to be perfect.

Motivation:
The existing work of anytime control [7]- [12] cannot effectively handle the fully distributed networked control scenario, where both the S-C and C-A communication links are wireless. Moreover, the existing research only considered simple wireless channel models, which cannot capture the key features of practical wireless channels that are time-varying and correlated [14]. Therefore, anytime control design and analysis for the fully distributed networked control system in practical wireless channels are critical in practice, but also present new challenges.
Novelty and contributions: In this article, we consider for the first time the sequence-based anytime control of a wireless networked control system (WNCS) in a generalized dual imperfect channel, where the S-C and C-A channel states are spatial-and time-correlated and are modeled as a multistate Markov process. Different from most of the existing works [9]- [12], where only the controller has a buffer to keep the calculated sequence of control inputs, we propose to use anytime control with buffers at both the controller and the actuator nodes. The latter is used to compensate for dropouts in the C-A channel. Moreover, the available computational resource of the controller is allowed to be time-correlated, modeled as a multistate Markov process. Such a dualchannel dual-buffer anytime control system has practical advantages but brings significant challenges to its analysis due to the complex system state updating rule, compared to previous setups. We propose a novel cycle-cost-based approach to derive sufficient conditions for stochastic stability of the overall WNCS. Our stability conditions are stated in terms of plant dynamics, network dynamics, buffer properties, and computational resource dynamics. We further show that, under suitable assumptions, the conditions derived guarantee robust stability when plant disturbances are taken into account.
The remainder of this article is organized as follows. Section II presents the system model of anytime control in the dual-channel-dualbuffer WNCS. Section III develops the stability condition. Section IV provides robust stability analysis. Finally, Section V concludes this article.
Notation: Sets are denoted by calligraphic capital letters, e.g., A. A\B denotes set subtraction. Matrices and vectors are denoted by capital and lowercase upright bold letters, e.g., A and a, respectively.

II. ANYTIME CONTROL IN A DUAL-CHANNEL DUAL-BUFFER WNCS
We consider a WNCS consisting of a plant system, a remote controller, and a wireless network placed between the plant and the controller. A sensor sends its measurements of the plant to the controller, and the controller computes and sends control commands to a remote actuator via the wireless network, as illustrated in Fig. 1. The dual-wireless-channel (i.e., the S-C and C-A channels) setup is different from that in [9]- [11], which assumed either perfect transmissions in two channels or a single (imperfect) wireless channel from the sensor to the controller. Each sampling period of the plant, denoted by T s , is divided into four phases: S-C transmission, command computation, C-A transmission, and implementation of control, as illustrated in Fig. 2.
We consider a process-noise-free discrete-time nonlinear plant model where x(t) ∈ R ls and u(t) ∈ R lu are the plant state and the control input at time t, respectively. Note that a more practical model with process noise will be investigated in Section IV. As a consequence of stochastic computational resources at the controller and packet dropouts in the sensing and control channels (which will be described later in more detail), the plant may have to operate in open loop for arbitrarily long time intervals. This may lead to performance degradation and potential loss of stability. Thus, throughout this article, we will analyze the stability conditions of the considered system, where the stability is defined as follows.
Definition 1: The process-noise-free dynamical system (1) is stochastically stable, if for some ψ ∈ K ∞ , the expected value Our standing assumption is that the plant is globally controllable (in the idealized closed loop case).

A. Dual Markov Fading Channels
We consider wireless fading channels for the S-C and C-A transmissions [15]- [17], where wireless channel status varies with time due to multipath propagation and shadowing caused by obstacles affecting the radio-frequency wave propagation. The time-varying channel conditions can be modeled as Markov processes [14]. Furthermore, in practice, the two channel conditions can be correlated, named as spatial correlation, which is caused by the same environment obstacles [18].
For wireless packet transmissions, there exists a fundamental tradeoff between reliability, i.e., the packet drop probability, and the data rate, which determines the amount of information bits that a packet can carry [19]. For a fixed channel condition, increasing the data rate of a packet can lead to a higher packet drop probability. For a good channel condition, one can reduce the packet drop probability while maintaining the fixed data rate. Alternatively, one can increase the data rate while maintaining the packet drop probability [20]. In this work, we will keep the data rate fixed for the S-C transmissions, since there is nothing gained from aggregating past sensor measurements in the state feedback case. However, it is well known that sending control sequences can be beneficial to compensate for packet dropouts. To accommodate this in a fading channel environment, we allow C-A transmissions to contain packets of varying data rate. The rates depend on the channel condition and provide a guaranteed packet dropout probability. Thus, a longer control sequence can be transmitted to the actuator under a better channel condition with the same reliability.
is the packet drop probability function in terms of the channel power gain. Then, we assume that the joint C-A and S-C channel condition {(B, B )} N is a time-homogeneous Markov process, and the state transition probability is given as Remark 1: Our current channel model jointly considers both the spatial-correlated S-C and C-A channels, the time-correlated fading channel conditions, and variable data rate requirements. To the best of our knowledge, this has never been considered in the literature of WNCSs and is more general than existing models. For example, independent dual channels with i.i.d. packet dropouts were considered in [21], which is a special case of our model when the fading channels degrade to static ones and the channels' spacial correlation is perfectly canceled.

B. Anytime Control With Dual Buffers
When considering perfect transmission between the controller and the actuator, as in [9]- [11], the system only needs one buffer at the controller to store the computed control commands. If imperfect transmissions are taken into account, it is convenient to include a command buffer at the actuator to provide robustness against packet dropouts (see, e.g., [22]) for a general packetized predictive control method. 2 Clearly, the dual-buffer system introduces a more complex state updating process.
Let Λ c and Λ a denote the length of the controller's and the actuator's buffer, respectively. Then, the buffer state at the controller after its transmission phase is denoted as The buffer state at the actuator right after the C-A transmission but before the implementation of a control command, i.e., the precontrol buffer, is denoted as In general, the buffers b a (t) and b c (t) keep the calculated sequences of control command, and the buffer updating rules will be given in the following part.
The control input is the first element in the actuator's buffer, i.e., which can be treated as the previously predicted control command for the current time slot. The buffer state at the actuator right after the control implementation, i.e., the postcontrol state, is b a (t) S a b a (t) 2 Markovian communication and computational resources were not considered in [22].
where the buffer shift matrices are defined as Let N (t) denote the number of calculated tentative future control commands at time t. The process {N } N is a time-homogeneous Markov process with the transition probability The controller's operations are described as follows. 1) If γ (t)N (t) > 0, the controller has a new update from the sensor and is available for computation. In this case, it discards all the existing commands in its buffer and generates a sequence of N (t) control commands to control the plant in time slots t to (t + N (t) − 1). The sequence of tentative controls is The buffer state before transmission is written as ] .
Specifically, the controller calculates the control sequence based on the anytime control algorithm proposed in [9], which is rewritten as where κ(·) is defined in Assumption 1. Considering the C-A channel capacity and the actuator's buffer length, the controller transmits min{B(t), N(t), Λ a } commands to the actuator. If the transmission is successful, the buffer shifts by min{B(t), N(t), Λ a } steps. Otherwise, the controller erases its buffer. This is because the first computed control command u 1 (t) cannot be implemented in the current time slot t, and the rest of computed control commands, which are calculated based on the successful implementation of the first control command, become useless. It is clear that at time instances where the actuator has run out of buffer contents, we have b a (t) = 0 and u(t) = 0 = u 1 (t). For the case that b a (t) = 0, the predicted control input u(t) is equal to the currently calculated control command u 1 (t) only in the perfect process-noise-free scenario, and u(t) = u 1 (t) in general. In these two cases, if the plant is out of control in the previous time slot, i.e., b a (t − 1) = 0, the controller erases its buffer due to the same reason in case 1); otherwise, the controller sends the buffered commands to the actuator as much as it can, subject to the constraints of the C-A channel capacity and the actuator's buffer length. The actuator's operations are described as follows. 1) If γ(t) = 0, the actuator's buffer is shifted by one step, i.e., b a (t) = S a b a (t − 1), since the first command in the buffer of the previous time slot was used for control. 2) If γ(t) = 1 and γ (t)N (t) > 0, the actuator erases the previous commands and stores the received ones. This is because the newly calculated control commands are expected to perform better than the previous calculated commands, especially when the process noise has a large variance. 3 3) If b a (t − 1) = 0 and γ (t)N (t) = 0, no operation on the actuator's buffer is required as there is no new commands transmitted. 4) If b a (t − 1) = 0, γ (t) = 1, and γ (t)N (t) = 0, the actuator shifts its buffer by one step due to the same reason in item 1) and stores the received commands in the buffer right after the existing commands. Let λ c (t) and λ a (t) denote the effective buffer lengths at the controller and the actuator, respectively. Intuitively, λ c (t) and λ a (t) jointly determine the closed-loop performance of plant, as a larger λ c (t) and a larger λ a (t) indicate that the plant will be delivered tentative control values for a longer time. Based on the controller's and the actuator's operations, the actuator is not necessary to have a larger buffer than that of the controller. Thus, we assume that Λ a ≤ Λ c .
Let L(t) denote the number of tentative commands to be transmitted. Based on the controller's operations, L(t) can be written as otherwise.
(4) Then, the buffer-updating rules based on the controller's and the actuator's operations are 3 Note that when the number of tentative future commands in the newly received packet is less than the previous ones, we still need to erase all the previous commands, not just part of them. The reason is that each buffered control command at the actuator was calculated assuming the successful implementation of the previously calculated control command. Thus, if the first few control commands at the actuator's buffer are removed (i.e., cannot be applied for control), then the rest of the buffer is of little use. and b a (t)= (6) From (5) and (6), the updating rules of the effective buffer lengths λ c (t) and λ a (t) are

III. STABILITY OF THE ANYTIME CONTROL SYSTEM
Based on the anytime control method described in (4)-(6), and following the established stability analysis framework adopting stochastic Lyapunov functions [9]- [11], to investigate the stability condition of the system (1), we only need to focus on the plant events that the actuator runs out of control commands, i.e., λ a (t) = 0, ∀t ∈ N 0 . However, since the process {λ a } N 0 has an infinite memory and is not a Markov process, the methods in [9]- [11] are not directly applicable. Instead, we shall analyze the control system through the aggregated Markov process {Z} N 0 defined as where X c {0, 1, . . . , Λ c } and X a {0, 1, . . . , min{Λ a ,N }}. Assume that Z(t), ∀t ∈ N 0 , belongs to the finite set S {s 0 , s 1 , . . . , s S } with cardinality S. Different from [10], which only needs to analyze an aggregated process of two processes, we need to investigate the aggregation of five processes, where both {λ c } N 0 and {λ a } N 0 are correlated with {B, B , N} N 0 . Since the control process is divided by the open-loop events with λ a (t) = 0, we define K = {k n } n∈N 0 as the sequence of time steps with λ a (t) = 0. We name the time sequence between k n and k n+1 as the (n + 1)th cycle of the process, ∀n ∈ N 0 . Then, the number of time steps between consecutive elements of K is Without loss of generality, let the set S 0 {s 0 , s 1 , . . . , s S 0 } ∈ S with cardinality of S 0 denote the subset of S consisting of all the states with λ a (t) = 0, and hence, Z(k n ) ∈ S 0 , ∀n ∈ N 0 . In [9]- [11], the set S 0 has only one state. In our scenario, S 0 > 1 introduces more challenges in analyzing the process {k n } n∈N 0 . The state transition process of {Z} N 0 is illustrated in Fig. 3.
In what follows, we study the properties of {Z} N 0 and {Z(k n )} n∈N 0 and then analyze the stability condition. Note that wireless fading channel conditions are commonly modeled as IA Markov processes [14], [15], [23], [24]; thus, it is reasonable to consider an IA Markov process {(B, B )} N 0 ∈ B × B . Also, the computation availability {N } N 0 ∈ N is commonly modeled as an IA Markov process [10]. Since an aggregation of IA Markov processes is still an IA Markov process [25], it is reasonable to consider the IA Markov process {(B, B , N) For the initial state of {Z} N 0 , it is practical to assume that both buffers are empty.
Note that v i,j can be numerically calculated based on (2), (3), (7), and (8), though it does not have a closed-from expression due to the complexity introduced by the dual-buffer updating process (7) and (8).

B. Analysis of the Stability Condition
Similar to [9]- [11], the stability of the WNCS depends on the statistics of {Δ n } n∈N in (10), which denotes the time duration between consecutive open-loop events. Different from [9] and [10], in the considered case, the process {Δ n } n∈N is not i.i.d. For the WNCS with a single channel, in [11], an event-triggered setup was considered, leading to {Δ n } n∈N , which is not i.i.d. However, our current setup is different from [11]. In particular, {Δ n } n∈N is formed by the first return time of a set of states in S 0 rather than a single state (as in [11]). Therefore, the approach in [11] cannot be adopted directly. In the following, we propose a novel cycle-cost-based approach to obtain sufficient stability conditions.

Lemma 3: The plant is stochastically stable if
where and ρ and α were defined in Assumptions 1 and 2, respectively. Proof: From Assumptions 1 and 2 and Definition 1, the plant is stable if we can prove that By using Assumptions 1 and 2, we have Then, it is easy to obtain that From Lemma 3, to find the stability condition of the system, we only need to investigate the process {Ξ} N . However, {Ξ} N is not Markovian, whereas the underlying process {Z(k n )} n∈N 0 is. In the following, we investigate two stability conditions, with and without exploring the state transition properties of the underlying process. The results are stated in Theorems 1 and 2, respectively.
Before proceeding, we need the following technical lemma. Lemma 4: The following inequality holds: and D(l) andṼ are defined in Lemma 2.
Proof: From (13), it can be shown that and hence where and r i,j is bounded due to the fact that ∞ l=1 [D(l)] i,j < 1 and ρ < 1.
Using Lemmas 3 and 4, it is straightforward to have the following result.

Remark 2:
For the special case with S 0 = 1, i.e., there exists only a single state in S leading to no control being applied, Lemma 2 shows that the processes {Z(k n )} n∈N 0 are i.i.d. and so is the process {Δ n } n∈N . Such a special case is identical to that of [10], and we see that the stability condition in Theorem 1 reduces to which is identical to that of [10]. Remark 3: The sufficient condition in Theorem 1 is obtained by considering the worst-case scenario of the state transitions, i.e., considering the pair of states s i and s j such that it has the largest conditional expectation E[ρ Δ n+1 |Z(k n ) = s i , Z(k n+1 ) = s j ]. However, such a method does not take into account the state transition probabilities iñ V defined in Lemma 2. Thus, the sufficient condition in Theorem 1 is conservative.
The following lemma is needed to obtain a less conservative stability condition in Theorem 2.

Remark 4:
This stability condition has considered the effect of all the conditional expectations E[ρ Δ n+1 |Z(k n ) = s i , Z(k n+1 ) = s j ] and the state transition probability matrix of {Z(k n )} n∈N , i.e.,Ṽ. Using the Perron-Frobenius theorem [27], since U is nonnegative, Thus, the sufficient condition in Theorem 2 is less restrictive than that in Theorem 1.

C. Numerical Example for Stability Condition Calculation
We set the buffer lengths Λ c = Λ a = 2, the maximum number of calculated control commands per time stepN = 2, the maximum number of commands that can be transmitted via the C-A channel B = 2, the number of channel states of the S-C channelB = 2, and the packet drop probabilities in the two states as 0.2 and 0.01. We assume that ρ = 0. 8 Thus, the process Z(t) defined in (9) has 3 × 3 × 3 × 2 × 3 = 162 states. Using the state transition rules of λ c (t) and λ a (t) in (7) and (8), respectively, and the channel and computational resource state transition probabilities (2) and (3), the state transition matrix of Z(t) can be calculated. Due to the space limitation, the matrix is presented in [28] with 72 transient states (highlighted in yellow), and thus, Taking (15) and (16) By applying (17) and (18) into Theorems 1 and 2, respectively, we have

IV. ROBUSTNESS TO PROCESS NOISE
In this section, we investigate the stability condition of the following plant system with process noise where w(t) ∈ R ls is a white noise process, which is independent of the other random processes of the system. For ease of analysis, we consider uniform bounds and continuity as follows.
When considering unbounded process noise, the stability condition in Definition 1 cannot be satisfied. Thus, we consider the following stability condition in terms of the average cost [10].
Remark 5: Theorem 3 shows that the stability condition for the process-noise-free case holds for the process-noise-present one as well under Assumption 5, which is in line with [10]. Note that in the case of Markov jump linear systems, stability conditions for noise-free cases are equivalent to conditions for noisy cases as well (see [29,Th. 3.33, Ch. 3]).

V. SIMULATION RESULTS
We present simulation results of the proposed dual-buffer anytime control algorithm and the baseline single-buffer algorithm [10], where the S-C and C-A channels are modeled as multistate Markov chains. The system parameters are the same as in the numerical example in Section III. We consider an open-loop unstable constrained plant model of (19) in the presence of noise [11] x 1 (t) x 2 (t) = x 2 (t) + u 1 (t) −sat(x 1 (t) + x 2 (t)) + u 2 (t) + w 1 (t) w 2 (t) and the process noise w 1 (t) and w 2 (t) are independent zero-mean i.i.d. Gaussian and E[w 1 (t) 2 ] = E[w 2 (t) 2 ] = 0.1. We take the control policy as κ(x) = [−x 2 , 0.505sat(x 1 + x 2 )] . In Fig. 4, we show the simulation of |x(t)| with 800 time steps for both the dual-buffer anytime method and the baseline method. It is clear that the dual-buffer method significantly outperforms the baseline, especially when the initial process state x(0) is far from the origin. Therefore, adding the command buffer at the actuator can effectively overcome the effect of packet dropouts, leading to significant control performance improvement.

VI. CONCLUSION
In this article, we have studied an anytime control algorithm for the dual-channel dual-buffer WNCS with random computational resources over correlated channels. We have proposed a novel approach to derive sufficient conditions for stochastic stability in the case of nonlinear plant models with disturbances. The stability conditions are stated in terms of plant dynamics, network dynamics, buffer properties, and computational resource dynamics. Our numerical results have shown that the proposed dual-buffer anytime control system can provide significant control performance improvement when compared to the conventional single-buffer system as it effectively overcomes the effect of packet dropouts. In future work, we will consider optimal control sequence design. In addition, we may include an extension to large-scale WNCSs with multiple plants and controllers and that need to share network and communication resources.