{'fraction_non_alphanumeric': 0.035661555934128335, 'fraction_numerical': 0.012776831345826235, 'mean_word_length': 4.7178571428571425, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Compact Formulation of the First Evolution Equation for Optimal Control Computation

The first evolution equation is derived under the Variation Evolving Method (VEM) that seeks optimal solutions with the variation evolution principle. To improve the performance, its compact form is developed. By replacing the states and costates variation evolution with that of the controls, the dimension-reduced Evolution Partial Differential Equation (EPDE) only solves the control variables along the variation time to get the optimal solution, and its definite conditions may be arbitrary. With this equation, the scale of the resulting Initial-value Problem (IVP), transformed via the semi-discrete method, is significantly reduced. Illustrative examples are solved and it is shown that the compact form evolution equation outperforms the primary form in the precision, and the efficiency may be higher for the dense discretization. Moreover, in discussing the connections to the classic iteration methods, it is uncovered that the computation scheme of the gradient method is the discrete implementation of the third evolution equation, and the compact form of the first evolution equation is a continuous realization of the Newton type iteration mechanism.


I. INTRODUCTION
Optimal control theory aims to determine the inputs to a dynamic system that optimize a specified performance index while satisfying constraints on the motion of the system. It is closely related to the engineering and has been widely studied [1]. Because of the complexity, Optimal Control Problems (OCPs) are usually solved with numerical methods. Various numerical methods are developed and generally they are divided into two classes, namely, the direct methods and the indirect methods [2]. The direct methods discretize the control or/and state variables to obtain the Nonlinear Programming (NLP) problem, for example, the widely-used direct shooting method [3] and the classic collocation method [4]. These methods are easy to apply, whereas the results obtained are usually suboptimal [5], and the optimal may be infinitely approached. The indirect methods transform the OCP to a Boundary-value Problem (BVP) through the optimality conditions. Typical methods of this type include the well-known indirect shooting method [2] and the novel symplectic method [6]. Although be more precise, the indirect methods often suffer from the significant numerical difficulty due to the ill-conditioning of the Hamiltonian dynamics, that is, the stability of the costates dynamics is adverse to that of the states dynamics [7]. The recent development, representatively the Pseudo-spectral (PS) method [8], blends the two types of methods, as it unifies the direct computation of NLP and the indirect realization of BVP in a dualization view [9]. Such methods inherit the advantages of both types and blur their difference.
Theories in the dynamics and control field often enlighten strategies for the optimal solutions, for example, the non-linear variable transformation to reduce the variables [10] and the dynamic method to solve the unconstrained parameter optimization problems [11]. Recently, a Variation Evolving Method (VEM), inspired by the states evolution within the stable continuous-time dynamic system, is proposed for the optimal control computation [12]- [20]. The VEM is built upon the infinite-dimensional dynamics Lyapunov principle, and it synthesizes the direct and indirect methods from a new standpoint. The Evolution Partial Differential Equation (EPDE), which describes the evolution of variables towards the optimal solution, is derived from the viewpoint of variation motion to reduce the performance index, and the optimality conditions will be gradually met with theoretical guarantee. In Refs. [12] and [13], besides the states and the controls, the costates are also employed in developing the EPDE (also named the ZS first evolution equation), and this increases the complexity of the computation. In Refs. [14]- [19], the VEM that uses only the original variables is systematically developed. The costate-free optimality conditions are established and the ZS second evolution equation is derived. In Ref. [20], the third evolution equation, the compact form of the second evolution equation in essence, is proposed. The EPDE therein only concerns the control variables and hence relieves the computation burden. To enhance the performance with the similar manner, in this paper the compact formulation of the first evolution equation will be studied.
Throughout the paper, our work is built upon the assumption that the solution for the optimization problem exists. We do not describe the existence conditions for the purpose of brevity. Relevant researches such as the Filippov-Cesari theorem are documented in Ref. [21]. In the following, first the principle of the VEM is reviewed. Then the first evolution equation, which addresses typical OCPs with terminal constraints, is presented. The compact formulation stems from a more concise unconstrained functional, and its form is quite different from the primary form. Later, illustrative examples are solved to test the developed equation. We also discuss the connections of the evolution equations to the classis iteration numerical methods at the end.

A. Infinite-dimensional Lyapunov theory
The VEM is a newly developed method for the optimal solutions. It is enlightened from the inverse consideration of the Lyapunov dynamics stability theory in the control field [22]. As the start point of this method, the finite-dimensional Lyapunov principle is generalized for the infinite-dimensional continuous-time dynamics.

Definition 1:
For an infinite-dimensional dynamic system described by is called an equilibrium solution.
denote the vector function supremum norm.

Lemma 1:
For the infinite-dimensional dynamic system (1), if there exists a continuously differentiable functional : where c is a constant.
Therefore, the statement may be proved by showing that To show that ĉ c = , we use a contradiction argument. Suppose ĉ c > . By the continuity of V , there is where the set S is defined as The existence of γ is guaranteed because the continuous functional V has a maximum over the compact set S . Thus Since the right-hand side of Eq. (4) will eventually become smaller than c , the inequality contradicts the assumption of ĉ c > . ■

B. Principle of VEM
The VEM analogizes the optimal solution to the asymptotically stable equilibrium point of an infinite-dimensional dynamic system, and derives such dynamics to minimize a specific performance index that acts the Lyapunov functional. To implement the idea, a virtual dimension, the variation time τ , is introduced to describe the process that a variable like ( ) t u evolves to the optimal solution under the dynamics governed by the variation dynamic evolution equations, which may be presented in the form of the EPDE and the Evolution Differential Equation (EDE). Fig. 1 illustrates the variation evolution of the control variables in the VEM to solve the OCP. Through the variation motion, the initial guess of control will evolve to the optimal solution, and the optimality conditions will be gradually achieved.  (5) where the elements of the variable vector ( ) n t ∈ y belong to 2 0 [ , ] f C t t . 0 t and f t are the fixed initial and terminal time.
The EPDE and EDEs derived are [12][14] where the column vectors are the shorthand notations of partial derivatives, and K is a n n × To seek the optimal solution, we need to solve the EPDE and the EDEs with right definite conditions. Via the well-known semi-discrete method in the field of Partial Differential Equation (PDE) numerical calculation [25], those equations are transformed to the finite-dimensional Initial-value Problems (IVPs) to be solved, with the common Ordinary Differential Equation (ODE) integration methods. Note that the resulting IVPs are defined with respect to the variation time τ , not the normal time t .

A. OCP definition
In this paper, we consider the OCPs with terminal constraint that are defined as subject to the dynamic equation (13) where t ∈ is the time.  (15) where : n q × → g is a q dimensional vector function with continuous first-order partial derivatives. Find the optimal solution ˆ( , ) x u that minimizes J , i.e.

B. The primary formulation
The first evolution equation that solves Problem 1 are derived through a constructed unconstrained functional, which employs the classic optimality conditions as where is the costates, and q ∈ π is the Lagrange multipliers that adjoin the terminal constraint (15) in the classic adjoining method [26].
and the transversality conditions where H x and H u represent the partial derivatives of H . x f and u f are the Jacobi matrixes of f . t ϕ , ϕ x , t g , and x g are the right-dimensional first-order partial derivatives.
With the VEM, the EPDE and the EDEs derived from the unconstrained functional (17) (regarded as the Lyapunov functional) , and τ =+∞ π will satisfy the feasibility conditions (13)- (15) and the optimality conditions (18) will reach its equilibrium solution on condition that Eq. (24) may be adapted as which will also preserve the same solution.

Remark 3:
If in Problem 1 the terminal time f t is fixed, then the unconstrained foundational is formulated as and the resulting evolution equations may be Eqs. (22), (26), (29), and ( ) ( ) which is consistent with Eq. (31).

Remark 4:
If in Problem 1 the terminal states are free, i.e., g vanishes, then the unconstrained foundational is simplified as and the resulting evolution equations developed may be Eqs. (22), (25), (29), and (31) with terms regarding g removed.
Employing the primary form of the first evolution equation to seek the optimal solution, the anticipated variable evolving along the variation time τ , as depicted in Fig. 1, includes the control variables, the state variables and the costate variables. When we apply the semi-discrete method to solve the EPDE (22), the state, costate, and control variables all need to be discretized along the normal time dimension t .

C. The compact formulation
With the similar treatment in establishing the third evolution equation [20], we do not solve for the states and the costates from the variation motion while they are computed in terms of the state equation (13) and the costate equation (18) with the boundary conditions given by Eqs. (14) and (20), namely In particular, we have derived the explicit expression of ( ) t λ for Eq. (36) in Ref. [15], that is is the n n × dimensional state transition matrix from time point t to time point σ , and it satisfies f Correspondingly, there is Substituting Eq. (45) into Eq. (41) gives With Eqs. (43), (47), (48) and define  and for the Lagrange multiplier parameters π as δ δτ = − π π π K n (55) where K is a m m × dimensional positive-definite gain matrix, f t k is a positive gain scalar, and π K is a q q × dimensional positive-definite gain matrix.
Use the partial differential operator " ∂ " and the differential operator " d " to reformulate the variation dynamic evolution equations (53)-(55), we may get the following EPDE and EDEs, i.e., the compact form of the first evolution equation, as With the compact form evolution equation, the anticipated variable evolving along the variation time τ , as depicted in Fig. 1, merely includes the control variables. Use the semi-discrete method to solve Eq. (56), only control variables are discretized for the numerical computation. Although both the compact form and the primary form evolution equations are derived from similar unconstrained functional, their concrete expressions are quite different. Besides the reduced dimensionality, it is shown that the EPDE (56) requires the integration, and the differentiation, as displayed in the EPDE (22), may be avoided. In seeking solutions, this is advantageous to reduce the numerical error.

IV. ILLUSTRATIVE EXAMPLES
First a linear example taken from Xie [27] is solved.
Example 1: Consider the following dynamic system . Find the solution that minimizes the performance index with the boundary conditions The one-dimensional gain matrix K was set as 1 K = and the 2 2 × dimensional gain matrix π K was 1 0 0 1 The initial conditions were 0 ( , ) 0 u t τ τ = = and 0 0 0 . Using the semi-discrete method, the time horizon 0 [ , ] f t t was discretized uniformly with 41 points. Thus, a dynamic system with 43 states was obtained and the OCP was transformed to a finite-dimensional IVP. Note that in this discretization granularity, there will be 207 states for the primary form evolution equation. Huge difference. To solve the IVP, the ODE integrator "ode15s" in Matlab, with default relative error tolerance 1×10 -3 and default absolute error tolerance 1×10 -6 , was employed. The spline interpolation was used to obtain the control within the discretization points. For comparison, the analytic solution by solving the BVP is also presented.
[ ]  In Table 1 . It is shown that the precision is much higher with the compact form evolution equation, while it takes more time for the integration. Especially, in Ref. [20], we solved this example with the third evolution equation, a compact form as well. By comparison, it is found that between the two compact forms, the computation precision is similar while there is some difference in the computation time, on which the third evolution equation used shorter. This is mainly because of the less integration it required. Now we consider a nonlinear example with free terminal time f t , the Brachistochrone problem [28], which describes the motion curve of the fastest descending.
Example 2: Consider the following dynamic system , and 10 g = is the gravity constant. Find the solution that minimizes the performance index with the boundary conditions We solved this example with the primary form evolution equation in Ref. [12] before. Here it will be addressed via the compact form evolution equation. In the specific evolution equations (56)-(58), the gain parameters were set as , respectively. The weights were The initial conditions were set to be . We also discretized the time horizon 0 [ , ] f t t uniformly, with 101 points. Then an IVP with 104 states is obtained. We employed "ode15s" in Matlab to carry out the numerical integration for this transformed IVP with respect to τ while used "ode45" to solve the state equation and the costate equation with respect to t in Eqs. (35) and (36). In the integrator setting, the relative error tolerance and the absolute error tolerance were 1×10 -3 and 1×10 -6 , respectively. The spline interpolation was again used to get the control within the discretization points. For comparison, we computed the optimal solution with GPOPS-II [29], a Radau PS method based OCP solver.
The control solutions are plotted in Fig. 6, and the asymptotical approach of the numerical results are demonstrated. Fig. 7 presents the evolution profiles of the Lagrange multipliers π . They reach the optimal value of 0.1477 0.0564 Fig. 8 gives the states curve in the x y coordinate plane, showing that the numerical results starting from the vertical line approach the optimal solution over time. In Fig. 9, the terminal time profile against the variation time τ is plotted. The result of f t declines rapidly at first and then approaches the minimum decline time gradually. It is almost unchanged after τ = 35s. At τ = 400s, we compute that f t = 0.8165s, same to the result from GPOPS-II. In Table 2, the optimization results using the compact and the primary evolution equations respectively are also compared. Note that the optimal solutions from GPOPS-II are denoted by a hat "^", It is again shown that the precision of solutions via the compact form evolution equation is much higher, and in this example the computation time it used is much smaller. By the way, recall the results from solving the third evolution equation in Ref. [20], the difference is also mainly on the computation time while the precision is similar.

V. CONNECTIONS TO THE CLASSIC ITERATION METHODS
In this section, the relations between the VEM and the classic numerical iteration methods that solve OCPs will be discussed. We will analogize them by considering a parameter optimization problem with performance index

VI. CONCLUSION
The compact form of the first evolution equation is formulated for better performance. It only considers the evolution of the control variables along the virtual variation time, thus decreasing the dimension of the Evolution Partial Differential Equation (EPDE) in the primary form. Even with arbitrary initial conditions, its solution is guaranteed to ultimately meet the optimality conditions by the Lyapunov principle. Notably, the compact form evolution equation employs the integral instead of the differential in the right function, and this is advantageous in seeking solutions. Using the semi-discrete method to solve this equation, the scale of the transformed Initial-value Problem (IVP) is significantly reduced. From the illustrative examples, it is shown that the compact form outperforms the primary form in both the precision and the efficiency for the dense discretization. In particular, by comparison with the classic numerical iteration methods, we establish the connections that the computation scheme in the gradient method is the discrete implementation of the third evolution equation proposed previously, and the compact form of the first evolution equation is a continuous version of the Newton type iteration mechanism.