{'fraction_non_alphanumeric': 0.04910217702208804, 'fraction_numerical': 0.028921023359288096, 'mean_word_length': 4.0684563758389265, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 15, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Stabilization of Stackelberg Game-Based Control Systems

In this paper, we are concerned with the stabilizatbility of Stackelberg game-based systems. In particular, two players are involved in the system where one is the follower to minimize the related cost function and the other is the leader to stabilize the system. The main contribution is to derive the necessary and sufficient condition for the stabilization of the game-based system. The key technique is to explicitly solve the forward and backward difference equations (FBDEs) based on the maximum principle and give the optimal feedback gain matrix of the leader by using the matrix maximum principle.


Introduction
In modern control theory, stabilizability is an essential and important concept, especially in system analysis and synthesis. During the past few years, there have been extensive studies. Brian D. O. Anderson, in 1971, had considered the stability of the time-invariant deterministic system in section 3.2 [1]. Under some additional assumptions, the stabilization of the closed-loop system was formed when the control law resulting from an infinite-time performance index [1]. Subsequently, the stabilization problem of control systems has been extended to stochastic systems, and then extended to time-delay systems, successively; see [2]- [7] and references therein. In [2], the stabilization of discrete-time systems with delay and multiplicative noise was studied. And it showed that the system was stabilizable in the mean-square sense if and only if an algebraic Riccati-ZXL equation had a particular solution.
The above mentioned results are all belong to classical control theory, that is, the controller has a certain objective to achieve, and the plant to be controlled does not have its own objective. However, in recent years, in the process of socio-economic modeling, it is often necessary to consider the strategic interactions between different decision makers, especially in order to better achieve the objectives of economic planning and policies. This leads to the emergence of the concept and method of stabilization policy theory in the dynamic game literature with hierarchical decision-making structure [8]. Moreover, with the popularization of the socialization of network information, it has become particularly important to consider the stabilization of cyber-physical systems for multiply decision makers [9]. In [9], it considered a resilient control problem where a linear dynamic system was subject to cyber attacks launched by correlated jammers in cyber-physical systems. It had proposed a novel leader-follower game formulation to model the interdependency between the operator and adversaries and devised a receding-horizon Stackelberg control law to maintain situational awareness, and further analyzed closed-loop system stability and performance. In [10], the infinite horizon linear-quadratic Stackelberg games for a discrete-time stochastic system with multiple decision makers was studied and necessary conditions for the existence of the Stackelberg strategy set were derived in terms of the solvability of cross-coupled stochastic algebraic equations. In [11], the linear-quadratic Stackelberg differential games including time preference rates with an open-loop information structure was investigated.
And sufficient conditions to guarantee a predefined degree of stability were given based on the distribution of the eigenvalues in the complex plane. By using a no-memory state feedback representation, [12] developed an incentive strategy for discrete time LQ state feedback Stackelberg games. It designed an incentive policy for the leader, while the followers rational reaction guaranteed system stability. [13] proposed an adaptive dynamic programming algorithm by solving the coupled partial differential equations and the Stackelberg feedback equilibrium solution was obtained to ensure the stability of the system according to the Lyapunov function. However, most works are focused on the stabilization of the Stackelberg strategy in infinite horizon.
Most recently, [14] introduced a new control framework called game-based control systems (GBCSs), which had a hierarchical decision-making structure. And [14] focused on the linear stochastic systems to give some explicit necessary and sufficient algebraic conditions on the controllability of the Nash equilibrium in GBCSs.
Motivated by [14], we study the stabilization problem generated by the leader for the two-player Stackelberg GBCSs, while the follower is designed to minimize its own cost function. In order to address the problem mentioned above, we consider the finite-time horizon open-loop Stackelberg strategy. We optimize the follower firstly, based on the maximum principle, we construct the non-homogeneous relationship between the costate and state and there derives a backward equation. By solving the FBDEs, we derive a homogeneous relationship between the state and non-homogeneous terms in costate equation. In the optimization of the leader, by using the matrix maximum principle, the optimization of the leader is converted into find the optimal gain matrix minimize the cost function and is subject to the state equation.
The rest of the paper is organized as follows. Section II presents some preliminaries of the stabilization problem for the two-player Stackelberg GBCSs. The finite-time horizon results are given in Section III. The stabilization of Stackelberg GBCSs is settled in Section IV. Numerical examples are given in Section V.
Conclusions are provided in Section VI.
Notation: R n denotes the n−dimensional Euclidean space and A T denotes the transpose of matrix A. A symmetric matrix M > 0 (reps. ≥ 0) means that it is positive definite (reps. positive semi-definite).

Problem Formulation
Considering the following discrete-time GBCSs where x k ∈ R n is the state, u 1 k ∈ R m1 and u 2 k ∈ R m2 are the control inputs of the follower and the leader, respectively. A and B i , i = 1, 2 are constant matrices of appropriate dimensions. The initial state is x 0 ∈ R n , and the cost function to be minimized by u 1 k is where Q 1 , R 12 are positive semi-definite matrices and R 11 is positive definite matrix of compatible dimensions.
Since (2.1)-(2.2) is the linear-quadratic system, we limit the leader's admissible control to the linear feedback strategies, i.e., the admissible control set of the leader is assumed to be U 2 , such that and the admissible control set of the follower is assumed to be U 1 .
Now, we are in the position to give the main problem we want to address.
Problem 1: Under the optimal controller u 1 k ∈ U 1 which minimizes the cost function J 1 , find the necessary and sufficient conditions for u 2 k = K 2 x k ∈ U 2 to stabilize the GBCS (2.1). What should be noted is that in order to fundamentally solve the Problem 1 mentioned above, we covert the problem of the stabilization of the GBCSs into a optimization problem based on the Stackelberg game, and denote the cost function of the leader as where Q 2 is positive semi-definite matrices, such that there exists some matrices C 2 satisfying Q 2 = C T 2 C 2 and R 21 , R 22 are positive definite matrices of compatible dimensions.

Finite-Horizon Results
It should pointed out that the controller to minimize (2.4) will be the best stabilization controller for (2.1). Now we will find the controller u 1 k and u 2 k to minimize (2.1) and (2.4), respectively. To this end, we firstly consider the following problem in finite-time horizon.

Problem 2:
Find the unique open-loop Stackelberg strategy (u 1 k , u 2 k ), subject to (2.1), where u 2 k is with state feedback form of u 2 k = K 2 k x k , and such that where To solve the Problem 2, we introduce the following Riccati equations with terminal value P 1 N +1 = H 1 and P 2 N +1 = H 2 , where 14) The optimal cost functions of the follower and the leader are Proof: The proof will be divided into three steps. The first step is to optimize the follower by the maximum principle. And the second step is to optimize the leader by the matrix maximum principle.
Finally, we will calculate the optimal cost functions of the two players.

The optimization of the follower
We consider the optimization of the follower. By using the maximum principle, we formulate the nonhomogeneous relationship between the state and the costate in this subsection.
To this end, define the Hamiltonian function of the follower as then according to the maximum principle, we have 1) leads to the relationship between λ k−1 and x k are no longer homogeneous. Thus, we assume that Adding (3.25) into the equilibrium equation (3.23), then it can be rewritten as thus, the control of the follower can be calculated as Next, we shall proof that (3.25) is established for any 0 ≤ k ≤ N by the method of inductive hypothesis, where P 1 k satisfies (3.5) and ζ k−1 satisfies the following equation, According to λ N = P 1 N +1 x N +1 and ζ N = 0, we can derive that (3.25) is established for k = N . Given any s ≥ 0, assume that (3.25) is established for any k ≥ s, where P 1 k+1 and ζ k satisfy (3.5) and (3.28), respectively, we shall show that (3.25) also holds for k = s − 1.
By using (3.25) and (3.24), then it derives which indicates that (3.25) holds for k = s − 1 with P 1 s and ζ s−1 satisfying (3.5) and (3.28), respectively. Following from (3.27), then the state (2.1) can be rewritten as Based on the discussion above, we have yielded the FBDEs (3.28) and (3.30) with initial value x 0 and terminal value ζ N = 0. Then, the optimization of the leader u 2 k is transformed into the optimization of the FBDEs.

The optimization of the leader
In this subsection, we consider the optimization of the leader. We will iteratively solve the FBDEs firstly.
And then, by using the matrix maximum principle, the optimal feedback gain matrix of the leader is yielded.
Adding u 2 k = K 2 k x k into (3.28) and (3.30), respectively, then there derives the FBDEs with the initial value x 0 and the terminal value ζ N = 0.
Then, by using inductive hypothesis, we will proof the FBDEs satisfy the homogeneous relationship and in this way, we have which satisfy (3.33) and (3.34) for k = N .
We take any n ≥ 0, and assume that ζ k−1 and x k+1 are as (3.33) and (3.34) for all k ≥ n + 1. We show that these conditions will also holds for k = n.
For k = n, it follows that then the state x n+1 is such that which is exactly (3.34) for k = n. And the costate equation ζ n−1 satisfies which is exactly (3.33) for k = n.
To this end, the control input of the follower can be written as Now, we are in the position to calculate the optimal feedback gain matrix of the leader.
Adding (3.37) and u 2 Denote the matrix as Then, combining (3.39) with (3.34), it derives that Hence, the optimization of the leader can be converted into finding the optimal feedback gain matrix K 2 k , minimizes the cost function (3.38) and satisfies (3.40).
To this end, denote the Hamiltonian function of the leader as applying the matrix maximum principle, it yields with terminal value P 2 N +1 = H 2 . According to the symmetric of the matrices X k and P 2 k+1 , we have, Since Γ 2 k+1 > 0, then the optimal gain matrix of the leader can be uniquely obtained, which satisfies (3.15). Based on the discussion above, we can derive that the optimal controllers of the follower and the leader are exactly (3.17) and (3.18).
By using (3.15), the Riccait equation of the leader can be written as which is exactly (3.6), with terminal value P 2 N +1 = H 2 . Lastly, we will calculate the optimal cost functions of the two players. We shall calculate the optimal cost function of the follower firstly.
By applying (2.1) and (3.24), we get Adding the above equation from k = 0 to k = N , we have And by using (3.25), (3.28) and (3.30), it yields Combining (3.47) with (3.48), thus, we have According to (3.34), we can yield that (3.50) Adding it into (3.33) and u 2 k = K 2 k x k , there follows Then the optimal cost function of the follower is Finally, we will consider the optimal cost function of the leader. Adding into the state (2.1), we have Denote the value function of the leader as Combining (3.55) with (3.56), the value function can be written as where the last equality is established because of Adding (3.57) from k = 0 to k = N on both sides of the equation, it yields that according to the optimal control of the leader is u 2 k = −(Γ 2 k+1 ) −1 M 2 k+1 x k , and compared with (2.4), the optimal cost function of the leader is exactly (3.20). This completest the proof of Theorem 1.

Stabilization Results
In this section, the stabilization of Stackelberg GBCSs will be investigated.
Before giving the main results, we introduce the following AREs: Throughout the rest of this paper, the following two standard assumptions are made, readers may refer to [5]. 2 ) is observable. Now, we will give the main results.
Theorem 2: Under Assumptions 1, 2 and if Υ is invertible, u 2 k ∈ U 2 stabilizes the GBCS (2.1) in stabilization if and only if there exists a unique positive definite solution P 1 and P 2 to the AREs (4.1)-(4.2). In this case, the stabilizing controller is given as while the optimal controller of minimizing J 1 is given as following, i.e., And the optimal cost functions are

.18)
Proof: We give the necessity proof at first.
Necessity: Under Assumption 1 and 2, and if Υ is invertible, suppose there exists controllers u 1 k and u 2 k with constants K 1 and K 2 such that u 1 k minimizes the cost function J 1 and u 2 k stabilizes the system (2.1) in stabilization. We will show that there exists a unique solution P 1 > 0 and P 2 > 0 are the solutions to the AREs (4.1) and (4.2), respectively.
To make the time horizon N explicit in the finite horizon case, we rewrite P 1 Since P 1 k (N ) satisfies the standard Riccati equation under the Assumption 1, and based on the monotone bounded theorem, we can yield that Next, we will show P 2 > 0.
Rewritten the Riccati equation (3.6) as According to the terminal condition H 2 = 0, then by using induction method, we have P 2 k (N ) ≥ 0. Following from (3.20), it derives that, the arbitrary of x 0 implies that P 2 0 (N ) ≤ P 2 0 (N + 1), i.e., P 2 0 (N ) increases with respect to N . Next the boundedness of P 2 0 (N ) is to be proved as below. Since the controllers u 2 k = K 2 x k stabilizes system (2.1), while u 1 k = K 1 x k minimizes the cost function (2.2), thus we have and then we can yield that Following from [15], it follows that there exist constant c 1 satisfying: Therefore, noting from (2.4) that Q 2 , K 1T R 21 K 1 and K 2T R 22 K 2 are both bounded, there exists constant Thus for any N > 0, from Theorem 1 we know that What should be noted is that we do not use the convergence of the T k (N ) in the proof of the convergence of P 2 k (N ), however, according to the Riccati equation (3.6), the existence of the convergence for P 2 k (N ) means that T k (N ) is also convergence, thus, we assume lim N →∞ T k (N ) = T .
Next, we will show that there exist a positive integer N 0 such that P 2 0 (N 0 ) is positive definite. Suppose this is not the case. Since P 2 0 (N ) ≥ 0, then we can get an non-empty set By using (4.21), which means that i.e., X N +1 ⊆ X N . Each X N is a non-empty finite-dimensional set, so where dim represents the dimension of the set. Thus, there must exist N 1 , such that for any N ≥ N 1 , which yields that X N = X N1 , and thus N ≥0 So there exists a nonzero vector x ∈ X N1 , such that Let x 0 be equal to x. Then the optimal value of (2.4) is as where x * k , u 1 * k and u 2 * k represent the optimal state and the optimal controllers, respectively. Note that R 21 > 0, R 22 > 0 and Q 2 = C T 2 C 2 ≥ 0. It follows that: The observability of (A, Q 1 2 2 ) given in Assumption 2 indicates that x 0 = x = 0, which is a contradiction with x = 0, i.e., there exists N 0 such that P 2 0 (N ) > 0 for N > N 0 . And we will show the uniqueness in the following paragraph.
Since P 1 > 0 satisfies the standard Riccati equation (4.1), then P 1 is uniquely solvable. Suppose Z 2 is another solution to (4.1), (4.2) and (4.3)-(4.12) satisfying Z 2 > 0, i.e., According to the uniqueness of the optimal cost function (4.16), then we can obtain Since x 0 is the arbitrary initial state, thus (4.40) implies that P 2 = Z 2 , the uniqueness of the solution to (4.2) has been obtained.
Sufficiency: Under Assumptions 1 and 2, and if Υ is invertible, suppose P 1 > 0 and P 2 > 0 are the solutions to (4.1) and (4.2), respectively, we shall show that u 1 k ∈ U 1 is to minimize the cost function J 1 and u 2 k ∈ U 2 stabilizes the system (2.1) in stabilization and minimizes the cost function J 2 . Denote the Lyapunov functionH 2 k asH By using (4.13) and (4.14), we have adding (4.42) into (2.1), it derives Then, combining (4.41) and (4.43), there follows where u 2 k = K 2 x k for k ≥ 0 has been imposed in the last identity. Thus, we can conclude thatH 2 k is monotonically decreasing with respect to k. And Since P 2 > 0, then H 2 k ≥ 0 is bounded below. According to monotone bounded theorem,H 2 k is convergent. We will show lim k→∞ x k = 0 in the following paragraph.
Actually, let m be any nonnegative integer, by adding from k = m to k = m + N on both sides of (4.44) and taking limitation of m, it holds that Taking limitation on both sides of (4.46) and using (4.45), we know that According to (4.22), we have Therefore, the controllers (4.13) stabilizes system (2.1) in stabilization.
To complete the proof the Theorem 2, we shall show that the optimal controllers (4.14) in Theorem 2 minimizes the cost function (2.2).
Thus, the optimal cost function of (2.2) is which is exactly (4.15).
And applying similar procedure to the Lyapunov function of H 2 k in (4.41), the controller (4.13) minimizes the cost function (2.4) can be immediately obtained by (4.44), and the optimal cost function is exactly (4.16).
This completes the proof.

Conclusion
In this paper, the stabilization of Stackelberg GBCSs is considered, where the follower is designed to minimise its own cost function and the leader is designed to stabilize the system. To solve the stabilization problem, we consider the finite-time horizon open-loop Stackelberg strategy firstly. Using the maximum principle, we optimize the follower firstly and find that there exist an non-homogeneous relationship between the state and the costate equation. By iteratively solving the FBDEs, the homogeneous relationship between the state and the inhomogeneous terms in costae is obtained. And then by the matrix maximum principle, the optimization of the leader is converted into find the optimal feedback gain matrix, which minimize the cost function of the leader and is subject to the state. Finally, the necessary and sufficient condition for the stabilization generated by the leader is derived.