{'fraction_non_alphanumeric': 0.035253290448178085, 'fraction_numerical': 0.010537898345253713, 'mean_word_length': 4.414642529789184, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 15, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
COMPOSING MPC WITH LQR AND NEURAL NETWORKS FOR EFFICIENT AND STABLE CONTROL∗

Model predictive control (MPC) is a powerful control method that handles dynamical systems with constraints. However, solving MPC iteratively in real time, i.e., implicit MPC, has been a challenge for 1) systems with low-latency requirements, 2) systems with limited computational resources, and 3) systems with fast and complex dynamics. To address this challenge, for lowdimensional linear systems, a classical approach is explicit MPC; for high-dimensional or nonlinear systems, a common approach is function approximation using neural networks. Both methods, whenever applicable, may improve the computational efficiency of the original MPC by several orders of magnitude. The existing methods have the following disadvantages: 1) explicit MPC does not apply to higher-dimensional problems or most of the problems with nonlinear constraints; and 2) function approximation is not guaranteed to find an accurate surrogate policy, the failure of which may lead to closed-loop instability. To address these issues, we propose a triple-mode hybrid control scheme, named Memory-Augmented MPC, by combining an efficient linear quadratic regulator, an efficient neural network, and a costly, fail-safe MPC. From its standard form, we derive two variants of such hybrid control scheme: one customized for chaotic systems and the other for slow systems. We prove stability of the control scheme with any arbitrary neural networks and test its computational performance in simulated numerical experiments.

1. Introduction. Model predictive control (MPC) is a powerful method for controlling dynamical systems with constraints [25,4]. It has been widely applied in robotics, ranging from ground [5] and aerial [1] vehicle control to humanoid [16] and quadruped [3] robot control.
To implement MPC, one has two conventional approaches: 1) implicit MPC, which solves an optimization problem in real time with a preferably efficient problem formulation and numerical scheme, 2) explicit MPC, which caches every solution of the optimization problem offline and looks up the cached solutions online.
Explicit MPC tends to have faster running time than implicit MPC. However, existing explicit methods have limited use cases. Common explicit MPC methods only handle MPC with quadratic cost and linear constraints. Within the set of applicable problems, explicit MPC does not work well for MPC of many variables and complex constraints, since caching solutions of these problems tends to demand a prohibitively large amount of memory.
Alternatively, one can approximate an MPC with a neural network (NN) through supervised learning. It develops a surrogate model of MPC by fitting a neural network into a trajectory dataset generated using implicit MPC. A well-trained neural network model behaves similarly to the original MPC model and can achieve similar control effectiveness at a fraction of the MPC's computational budget. Nevertheless, to our best knowledge, we find the NN function approximation approach has yet to fully address the following open problems: 1) it is difficult, if not impossible, to guarantee that the approximation will always converge to a solution of least approximation errors; 2) it is challenging to verify that the approximation will produce a stable closed-loop dynamics without any knowledge of approximation errors between the original and the approximated control laws.
Motivated by the shortcomings of explicit MPC and NN function approximation, we propose a triple-mode hybrid control scheme, named Memory-Augmented MPC (MAMPC). The core idea is to mix a costly MPC controller with an efficient linear quadratic regulator (LQR) controller and an efficient NN controller, whenever stability permits. We summarize the main contributions of our work below.
• We present a novel controller design, i.e., Memory-Augmented MPC, in its standard form and two modified forms. • We prove stability of Memory-Augmented MPC without imposing any condition on the approximation errors of the neural network. • We demonstrate computational performance of Memory-Augmented MPC via four simulated robotic control applications. The rest of the article is organized as follows. A brief literature review is provided in Section 2 to situate the work in context. Terminology and notations of MPC are introduced in Section 3. Standard and modified versions of MAMPC are presented in Section 4. Theoretical properties of MAMPC, including stability, are proven in Section 5. Four numerical experiments are presented in Section 6 and discussed in Section 7. Lastly, major findings and future work are briefed in Section 8.
2. Related Works. A major limitation of MPC has been a lack of computational efficiency as evident in many robotic applications such as the Atlas humanoid robot [16] and the MIT Cheetah robot [3]. To overcome this shortcoming, we have three basic approaches: 1) developing an efficient implicit MPC policy, 2) precompiling an explicit MPC policy, and 3) approximating the MPC policy by an efficient surrogate policy such as a NN.
Implicit MPC approach implements the MPC policy by developing an efficient numerical scheme for solving the optimization program. This is perhaps the most common approach of implementing a MPC policy. Efficient implicit MPC controller often depends on custom optimization solvers that leverage structures specific to the problems they solve, such as [8,18]. Implementation of such implicit MPC is usually done in a low-level programming language such as C [10,19]. Despite of being computationally expensive, recent development in algorithms and hardware enables some implicit MPC controllers to achieve high-frequency performance of approximately 1000 Hz [9,15].
When existing implicit MPC methods do not provide satisfactory latency, one sometimes resorts to explicit MPC, whenever such alternative is applicable. Explicit MPC approach implements the MPC policy by precomputing the solution of the optimization program and looking up the solution in real time. At its core, explicit methods achieve reduced running time at the cost of increased memory footprint. To this end, numerous works has been developed to reduce its space complexity [27,17]. Nevertheless, due to poor scalability and lack of theory on nonlinear systems, explicit MPC has limited use cases in large-scale robotic applications.
When the explicit MPC method fails to apply to a problem due to scalability or nonlinearity, one may consider another alternative, namely, function approximation. Function approximation approach implements a MPC policy by approximating offline that MPC control law with an efficient surrogate model, such as a neural network, and uses that surrogate model for online deployment. Early works in this direction include [23,22], which has described a procedure to design a surrogate NN controller and has proposed conditions to guarantee its closed-loop stability. Later development has focused on guaranteeing different theoretical properties of the NN-controlled closedloop system. For example, [30] proposes a NN method with probabilistic optimality bounds; [24] has designed a projection operator that can modify any arbitrary NN controller to a stabilizing one; [12] proposes a mixed-integer linear program to certify stability and feasibility of ReLU-activated NN controllers; and [11] has developed a method that guarantees certain performance indicator of a NN-controlled system using probabilistic validation methods. Besides, rather than replacing the original MPC controller, a very recent work [7] uses a NN to warm start an optimization solver, which can then solve the MPC policy faster in real time. For a comprehensive study, readers may consult [31]. Applications of function approximation methods have found successes in practice, such as [21].
It is worth noting that closed-loop stability of the function approximation methods above, except [24], requires reasonable convergence of NN model to the original control law. Nevertheless, training a NN is commonly a high-dimensional nonlinear program. This nonlinear program is solved with faith by a black-box solver, such as [14], which in general does not come with any guarantee on convergence or optimality. Hence, for most of the existing works, their closed-loop stability replies in large part on the faithful assumption that there exists a black-box optimization procedure that can somehow find a good approximation to the original MPC.
In contrary, we propose a novel approach named Memory-Augmented MPC, which can work without any condition on the convergence and optimality of the NN. Compared to the technique in [24], which only applies to discrete-time linear time-invariant plants, our method also works for general nonlinear plants. Concretely, the hybrid controller combines a NN with a costly MPC and an efficient LQR. This hybrid design is an extension of the early dual-mode MPC design [20], which just combines a MPC with a LQR. An advantage of our approach over [20] is that it may provide more reduction in latency owing to the addition of a fast intermediate NN mode. As presented in Section 7, with parallelization, MAMPC is at worse equivalent to [20].
We continue the study by next introducing a basic set of terminology and notations in Section 3.

A Primer on MPC.
We consider the discrete-time time-invariant constrained dynamical systems of the form is the input at time index i, f : R n × R m → R n is continuously differentiable in both its components, with an equilibrium point at (0 n , 0 m ), i.e., f (0 n , 0 m ) = 0 n , and the linearized time-invariant system of f (·, ·) around the equilibrium is stabilizable. Let X be state constraint set and U be input constraint set. We consider system constraints of the following form A := X×U. The system dynamics is a map f : A → X and any control law is a map u : X → U.
Consider a finite planning and control horizon N . Let Provided with the dynamical system (3.1), a MPC is an optimal control law implicitly defined through the following set of optimization problems where N is planning and control horizon, c : A → R + is a continuous stage cost function, c f : ) are two sets of optimization variables corresponding to predicted states and anticipated inputs, X f is a terminal constraint set containing the origin usually designed to guarantee asymptotic stability. If optimization problem (3.2) is feasible and has an optimal solution (X 1:N * i , U 0:N −1 * i ), then the MPC control law selects the first element of U 0:N −1 * i as the input at time index i, that is, This process is repeated at the next time index i + 1, until some termination criterion is met. As a result, we obtain a closed-loop system 2) is feasible} is the set of all admissible initial states. For well-posedness of MPC, we require X ⊆ be the optimal cumulative cost. By Lyapunov stability theorem, we can derive the following sufficient condition for asymptotic stability of MPC as in Theorem 12.2 in [4].
then the MPC problem (3.2) is persistently feasible and the closed-loop system (3.4) is locally asymptotically stable with respect to the origin. In addition, J * (·) is a Lyapunov function and X 0 is a positively invariant region of attraction.
With the above definitions and theorem, we are ready to describe the composition rules of MPC, LQR, and NN in Section 4.

4.
Composing MPC, LQR, and NN. Provided with a computationally costly implicit MPC u MPC that satisfies Theorem (3.1), we propose a hybrid control scheme, namely, Memory-Augmented Model Predictive Control (MAMPC), u MAMPC : X → U, by augmenting u MPC with a LQR controller u LQR and a NN controller u NN , where the LQR controller and the NN controller are independently developed from the implicit MPC. In addition, we present two variants of MAMPC specifically tailored for chaotic systems and slow systems, respectively. 4.1. LQR from MPC. Provided with a MPC, we derive an infinite-time LQR controller by 1) linearizing the system dynamics around the equilibrium, 2) removing stage constraints, terminal constraint, and terminal cost, 3) taking planning and control horizon N to ∞, and 4) replacing stage cost c(·) with a positive definite quadratic cost, if it is not already so in the original MPC.
Formally, the MPC-induced LQR problem is defined as follows ∈ R n×m are linearized system dynamics around the equilibrium, and Q ∈ {N ∈ R n×n | N 0}, R ∈ {M ∈ R m×m | M 0} are weighting matrices that measure significance of state deviations and control costs, respectively. It can be shown that the optimal LQR control law for an initial state x is a linear map (4.2) u LQR (x) = −Kx for some K ∈ R m×n [2]. Furthermore, if the linearized system (A, B) is stabilizable and if A − BK is Hurwitz, then the closed-loop system is locally asymptotically stable near the equilibrium with a positively invariant region of attraction R LQR , as shown in Theorem 4.7 in [13]. Specifically, R LQR is a set that if x 0 ∈ R LQR and x[0] = x 0 , then x[i] ∈ R LQR , ∀i ∈ Z >0 and lim i→∞ x[i] = 0. As implied by local asymptotic stability, the largest possible R LQR is nonempty. Because the LQR controller only requires one matrix multiplication within a single controller step, it is usually significantly more efficient than the implicit MPC in terms of per-step computation. Hence, it is usually economical to switch to the LQR controller when the state is in the region of attraction of the LQR, that is,

NN from MPC.
Provided with a MPC, we derive a NN controller 1) by imitating the MPC policy u MPC (x) through supervised learning and 2) optionally by interacting with the environment through reinforcement learning, as illustrated in Figure 1. In the imitation phase, the NN training is a supervised learning problem usually of the following mean squared form where φ : R n → R m is a NN model with weights W and D is a set of M states randomly sampled from X 0 . Therefore, the resulting NN control law is where W is a suboptimal solution to problem (4.4), generally estimated via a gradient method such as Adam [14].
In the adaptation phase, it is possible to fine tune the NN further through a reinforcement learning (RL) algorithm. The training setup varies based on the specific choice of the RL algorithm. A comprehensive review of reinforcement learning is clearly beyond the scope of this work. Interested readers may refer to the standard text on this topic [28]. Lastly, we remark that this phase may be skipped if the NN trained in the imitation phase is sufficiently effective for the target control application or if the RL algorithm does not bring any improvement to the trained NN.
In practice, people find that implicit MPC controllers can be well approximated by a more efficient NN controller [23]. However, it is not easy to certify stability of the closed-loop system without bounding approximation errors between the NN controller and the original MPC controller.
To solve the problem, we show that by composing the NN controller with the MPC and the LQR in a hybrid control scheme, we can prove the local asymptotic stability of the closed-loop system at an acceptable cost of computational efficiency, even if the NN is random. Next, we present the basic form of our method, i.e., the standard MAMPC.

Standard
Memory-Augmented MPC. The hybrid control scheme combines the LQR controller and the NN controller with the original MPC controller. At every control step, if the state is in the region of attraction of the LQR, apply the LQR controller; else, we simulate the closed-loop system (4.6) for up to N LQR steps: if there exists a step j ≤ N LQR such that the state reaches within the region of attraction of LQR and that up until the j th step the simulated system does not violate any stage constraint of the MPC, apply the NN controller; otherwise, if the state is within the admissible initial states of the MPC, apply the implicit MPC.
Formally, the hybrid controller is defined as follows where y is the simulated state by numerically stepping: A pseudocode of the u MAMPC (x) is described in Algorithm 4.1. As we show in Section 5, the stability of NN relies on verifying whether the system reaches within R LQR through forward simulation. However, this approach is ineffective for systems that are sensitive to initial conditions and systems that require a large number of steps to converge to the origin. To address these two challenges, we introduce the following two variants of MAMPC.

Alternating-Authority
Memory-Augmented MPC. The first variant of MAMPC is designed specifically for chaotic systems. A chaotic system is loosely defined as a system that is very sensitive to initial condition and control input. Qualitatively speaking, the same chaotic system that begins with two slightly different initial conditions and control sequences will arrive at significantly different terminal states. As a result, it is very challenging to stabilize a chaotic system with any NN controller because the function approximator will inevitably produce random control errors which can easily deter the system from its stabilizing trajectory. In this case, it is useful to modify the hybrid control scheme to bound error accumulation induced by the NN by periodically alternating between the NN controller and the MPC controller. We name this modified version of MAMPC as alternating-authority MAMPC, which is formally described below.
The main variation from the standard MAMPC is that we do not verify if the system falls within R LQR in N LQR steps, as forward simulation is not as reliable for chaotic systems. This implies that i d should be chosen as a small positive number. Otherwise, prediction produced by forward simulation will be a reliable estimate of the future. However, note that reducing i d will prolong running time. Therefore, a rule of thumb of designing i d should be to choose it as large as the stability permits.
In addition, we remark that the closed-loop system has a smaller region of attaction compared to standard MAMPC, since we do not verify whether the system enters R LQR through forward simulation.

Way-Point Memory-Augmented MPC.
Another variant of MAMPC is designed specifically for slow systems. A slow system is loosely defined as a system that requires a substantial number of steps to steer close to the origin. For slow systems, it is difficult to choose an effective verification horizon N LQR : a small N LQR may be too short to predict convergence of the NN controller; a large N LQR may incur computational overhead and thus render the hybrid control inefficient. To address this problem, we propose a second variant of MAMPC by introducing a "way-point" set, D WP , and enabling the NN controller to be applied as long as the anticipated trajectory falls with that way-point set. We name this variant of MAMPC as way-point MAMPC, which is formulated as follows. if u NN steers f into D WP within N WP steps without violating A then 11: u ← u NN (x) 12: else 13: u ← u MPC (x) 14: end if 15: end if 16: return u At minimum, we require that the way-point set contains the region of attraction of the LQR, that is, R LQR ⊂ D WP . Additional improvement of the way-point set design can be performed through search. For example, one can parameterize D WP with one or more parameters and search for a set of parameters that produces the most efficient closed-loop performance.
Besides, note that introduction of a way-point set may reduce the area of the region of attraction of the closed-loop system.
With the three MAMPC hybrid control schemes defined above, we are ready to analyze their theoretical properties in Section 5.

Theoretical Analysis.
In this section, we prove the stability of the three MAMPC methods proposed above. In addition, we remark on robustness under model uncertainties and necessity of the fail-safe mode.

Stability of MAMPC.
We show that the three MAMPC methods are locally asymptotically stable with slightly different stability properties. We first prove that standard MAMPC is locally asymptotically stable in X 0 , as stated in the theorem below. . Furthermore, the closedloop system Proof. Existence of MAMPC depends on existence of LQR and NN. A LQR controller of the form (4.1) can always be derived from (3.2) because 1) f is continuously differentiable, so it can always be linearized to produce the A, B matrices in (4.1); 2) the linearized system (A, B) is by assumption stabilizable, so region of attraction of LQR is nonempty, i.e., R LQR = ∅; 3) as for Q, R we only require Q 0, R 0, which is always possible; 4) lastly, optimization problem (4.1) is just a relaxation of problem (3.2), which can always be solved.
Meanwhile, a NN controller trivially exists because we do not require NN to satisfy any properties other than the basic definition of a NN. This completes the first part of the proof on existence.
To prove local asymptotic stability of standard MAMPC in X 0 , we partition X 0 into three regions as follows where R NN is the set of states where the NN is invoked and R MPC is the rest of states in X 0 , i.e., R MPC := X 0 \ (R LQR ∪ R NN ). By construction, R LQR , R NN , and R MPC are mutually disjoint.
For every state x[0] ∈ R LQR , u LQR is invoked and the closed-loop system (5.1) will be equivalent to (4.3), which is locally asymptotically stable.
For every state x[0] ∈ R NN , u NN is invoked and the closed-loop system (5.1) will be equivalent to (4.6) until the state enters R LQR in no more than N LQR steps. The system will stay in R LQR and will be taken to the origin by u LQR as i → ∞. Consequently, the closed-loop system (5.1) is locally asymptotically stable in R MPC ∪ R NN .
For every state x[0] ∈ R MPC , u MPC is invoked and the closed-loop system (5.1) will be equivalent to (3.4) for at least one time step. In the next time step, if the state enters R LQR ∪ R NN , then the closed-loop system will converge to the origin as it is locally asymptotically stable in R LQR ∪ R NN ; otherwise, R MPC will be invoked again. Because the closed-loop system (3.4) is locally asymptotically stable, even if MPC never brings the state into R NN , it will eventually steer the system into R LQR in finite time, which will then be taken to the origin by u LQR . Consequently, the closed-loop system (5.1) is locally asymptotically stable in X 0 .
This completes the second part of the proof on stability.
To provide geometric intuition on how a MAMPC achieves local asymptotic stability, we illustrate schematically in Figure 2a how a MAMPC could steer a system from an initial state to the equilibrium at zero. A state in R MPC is first steered by u MPC into R NN , which is then steered by u NN into R LQR , after which is steered to zero by u LQR . Note that the example demonstrated in Figure 2a is only a hypothetical particular case of MAMPC closed-loop behaviors.
Next, for alternating-authority MAMPC, the closed-loop system no longer has guaranteed local asymptotic stability in X 0 , because it does not check whether NN shoots inside R LQR . Rather, we can only show that the system is locally asymptotically stable in R LQR and will never escape X 0 , as stated the theorem below. Furthermore, the closed-loop system Proof. The first part of the proof on existence is identical to the one in Theorem 5.1.
To prove stability of alternating-authority MAMPC, we partition X 0 into two disjoint regions as follows For every state x[0] ∈ R LQR , the closed-loop system (5.2) is identical to closedloop system (5.1), which is locally asymptotically stable, as shown in Theorem (5.1).
Next, we prove by contradiction that the system will never escape X 0 . Suppose there exits an escaping controlū such thatx ∈ X 0 but f (x,ū) / ∈ X 0 . This escaping controlū must be produced by either u MPC , u NN , or u LQR , that is,ū = u MPC (x), u = u NN (x), orū = u LQR (x). Ifū = u MPC (x), then u MPC is not persistently feasible in X 0 , which is not possible because we assume that u MPC satisfies Theorem (3.1). Ifū = u NN (x), then the following switch condition must hold: f (x,ū) ∈ X ⊆ X 0 . However, this is not possible sinceū is assumed to be an escaping control, that is, f (x,ū) / ∈ X 0 . Ifū = u LQR (x), then there must bex ∈ R LQR . Since R LQR is positively invariant, we must have f (x,ū) ∈ R LQR ⊂ X 0 , which is not possible since f (x,ū) / ∈ X 0 . Therefore, the system will never escape X 0 . This completes the second part of the proof on stability.
An illustration of how the alternating-authority MAMPC could steer a system to the equilibrium is provided in Figure 2b. Note that marginal stability may be an understatement in practice: a well-trained NN, combined with a modest alternation period i d , could very well result in a closed-loop system that is asymptotically stable.
If analytical guarantee of asymptotic stability is absolutely required, one can replace the admissible control set X 0 with a time-varying set S[i], where for some n ∈ Z >1 with S[0] = X 0 and S[n] = R LQR . Geometrically, S can be viewed as a set that gets successively shrunk from X 0 toward R LQR . Such design will invoke the fail-safe MPC if the systems is "stuck" outside of R LQR .
Lastly, similar to alternating-authority MAMPC, the closed-loop system of a waypoint MAMPC is locally asymptotically stable in R LQR and will eventually enter and stay within a level set that escribes D WP , as stated in the following theorem. . Furthermore, the closedloop system where V : R n → R + is a Lyapunov function of the closed-loop system (3.4) and ∂D WP is the boundary of the compact set D WP which by construction contains R LQR .
Proof. The first part of the proof on existence is identical to the one in Theorem 5.1.
To prove stability of way-point MAMPC, we partition X 0 into five disjoint regions as follows where R i NN is the set of states in or on the boundary of D WP where the NN is For every state x ∈ R LQR , the closed-loop system (5.4) is identical to closed-loop system (5.1), which is locally asymptotically stable, as shown in Theorem (5.1).
Next, we prove that the system will eventually enter and stay within D + WP . For every state x[0] ∈ X 0 \ D WP , either NN or MPC will steer the system into D + WP in finite time. If NN is ever invoked, the system (5.4) will be taken into D WP in no more than N WP steps. Otherwise, MPC will bring the system (5.4) into D WP in finite time since the MPC is locally asymptotically stable in X 0 .
For every state x[0] ∈ D WP , we prove by contradiction that there exists a j ≥ 0 such that for all i ≥ j, then the close-loop system (5.4) is equivalent to system (3.4).
but this is not possible because V is a Lyapunov function of the system (3.4), i.e., , then by assumption lim i→∞ x[i] does not exist. However, this is impossible because lim i→∞ x[i] = 0 by the local asymptotic stability of the system (5.4) in R LQR . Consequently, the assumption must be false.
This completes the second part of the proof on stability.
An illustration of how the way-point MAMPC could steer a system to the equilibrium is provided in Figure 2c. As in the alternating-authority MAMPC, to always achieve asymptotic stability, one can successively shrink D WP until D WP = R LQR in the same way as presented in equation (5.3).

Remark on Robustness.
In some cases, MAMPC may be robustified to account for model uncertainties. To this end, we propose a robustification procedure to improve the robustness of MAMPC using the idea of bounding error margin. We demonstrate the procedure by applying it to the standard MAMPC.
Provided with the true system dynamics as f , we distinguishf as the model of ∈ R LQR , potentially leading to a failure of MAMPC. To address this problem, we define a set operator Ero δ as follows where X is an arbitrary set and ∂X denotes the boundary of the set X. We claim that for ∆x u [k|i] = α > 0, ifȳ[k] ∈ Ero δ (R LQR ) with δ ≥ α, then there must be y[k] ∈ R LQR . Proof of the claim is a direct application of the triangle inequality of norm. We can therefore incorporate the set operator Ero δ to enhance the robustness of MAMPC with respect to model uncertainties.
We can apply the above procedure to robustify the standard MAMPC as follows where R δ LQR := Ero δ (R LQR ) and A δ := Ero δ (X) × U, for some δ > 0. The stability of the above robustified MAMPC is stated in the following theorem.
Then the following closed-loop system Proof. The proof is identical to that of Theorem 5.1. Note that it possible that R δ LQR = ∅. In this case, the hybrid control is equivalent to a dual-mode MPC-LQR controller. Similar to the above example, the robustification procedure can also be applied to enhance the alternating-authority MAMPC and way-point MAMPC.

Remark on Necessity of Fail-Safe MPC.
The hybrid control scheme can clearly be more efficient if we can skip the forward verification and remove the fail-safe MPC mode. In light of this observation, we describe a condition, where the fail-safe MPC mode will never be invoked and thus can be safely removed from the hybrid control scheme.
Define the following "fail-free" MAMPC Define L : X → R + as an augmented cumulative cost function of NN, that is, From Theorem 13.1 in [4], we are ready to state the condition in the following theorem.
Theorem 5.5 (Fail-Free MAMPC). If there exists some nonnegative function γ : then the closed-loop system with x[0] = x 0 ∈ X 0 is locally asymptotically stable in X 0 without violating any constraints in the original MPC problem (3.2).
Proof. The proof of the theorem is a direct application of Theorem 13.1 in [4].
Note that inequality conditions (5.5) are typically verified through sampling and interpolation [4], which makes this approach only suitable for relatively simple, moderately sized problems. Besides, the above condition is only a sufficient condition, for which we can safely discard the fail-safe MPC mode. Other methods are possible. For example, [26,6] have proposed methods to verify the stability of a NN controller using Lyapunov approaches. Last but not least, we emphasize that the above analysis does not impose any condition on the approximation error of the NN. In fact, so long as the NN has the right input and output dimension, the corresponding MAMPC will be closed-loop stable, even if the weights of the NN are random. In the worst case, MAMPC reduces to a hybrid controller consisting of just the MPC and the LQR, where the NN mode will never be invoked. This reduced hybrid controller is identical to one proposed in [20].
Next, we evaluate the computational performance of the proposed method in Section 6.
6. Numerical Experiments. To evaluate the running time performance of MAMPC, we conduct numerical experiments on four models in robotics, i.e., pendulum, triple pendulum, bicopter, and quadcopter. The models are selected for their significance in robotic applications: pendulum models are the building blocks for robot arm and leg control; copter models are an important model class in unmanned aerial vehicle control. We highlight key parameters and results of the four experiments in this section. For implementation details, please refer to the source code [29].
6.1. Pendulum. The first model is a single arm pendulum as shown in Figure 3. The goal is to maintain the pendulum at the position of the highest potential energy as marked by the dashed line.
The state is defined as x := [θθ] ∈ [−π, π)×R, where θ is the angular displacement from the inverted position. The input is a scalar u ∈ R, which is the applied torque at the joint. The equation of motion is described by the following nonlinear dynamics where definitions and values of parameters are specified in Table 1. By convention, we assign θ to be the clockwise angular displacement between the rod and the dashed line.  The control objective is to steer the pendulum to the origin by applying a limited torque u ∈ [−0.05, 0.05] in N · m at the joint. The MPC design highlights a sampling interval of 0.1 s, a planning horizon N = 5, and three optimization variables per step, resulting in a linearly-constrained quadratic programming (LCQP) of 0.5 s prediction horizon and 15 variables. Plant nonlinearity is handled through linearizing around the equilibrium, leading to a linear time-invariant system. We apply a standard MAMPC to control the pendulum with The NN is a two-layer, 20-neuron multilayer perceptron (MLP) trained through supervised learning with data randomly sampled from θ ∈ [π, π],θ ∈ [−1, 1].
To evaluate the computational efficiency of MAMPC, we compare its per-step running time and total running time with implicit and explicit MPC. The system is simulated with the same initial condition at [π/2 1/2] with the three different controllers. The per-step running times of the three controllers are shown in Figure 4 with the corresponding statistics tabulated in the first column of Table 5. The total running times of the three controllers are summarized in the first column of Table 6. In addition, An up time analysis of the three control modes within the MAMPC is tabulated in the first column of Table 7.

Triple
Pendulum. The second model is a triple pendulum as shown in Figure 5, which extends the above pendulum model to a more complex use case. The goal is to maintain the triple pendulum at the position of the highest potential energy.
The state is defined as x := [θ 1θ1 θ 2θ2 θ 3θ3 ] ∈ ([−π, π) × R) 3 , where θ 1 , θ 2 , θ 3 are the angular displacements of the three links respectively. The input is defined as u := [u 1 u 2 u 3 ] ∈ R 3 , which is the applied torques at the three joints respectively. The equation of motion of the triple pendulum is obtained through symbolic computation using the principle of Lagrangian mechanics: where L := T − V is the Lagrangian of the system, T is the kinematic energy, and V is the potential energy. The kinematic energy and potential energy of the triple pendulum are where the coordinates of the three joints are calculated as follows x 1 = l 1 cos (π/2 − θ 1 ) , y 1 = l 1 sin (π/2 − θ 1 ) , x 2 = x 1 + l 2 cos (π/2 − θ 1 + θ 2 ) , y 2 = y 1 + l 2 sin (π/2 − θ 1 + θ 2 ) , where definitions and values of parameters are specified in Table 2. For simplicity, we assign θ 1 , θ 2 , θ 3 to be the counterclockwise angular displacements between the respective links and the perpendicular directions of the corresponding dashed lines. Explicit form of the system dynamics takes pages to write and is therefore omitted here.   The control objective is to steer the triple pendulum to the origin by applying three limited torque inputs u 1 , u 2 , u 3 ∈ [−1, 1] in N · m at the three joints, respectively. The MPC design highlights a sampling interval of 0.1 s, a planning horizon N = 5, and nine optimization variables per step, resulting in a LCQP of 0.5 s prediction horizon and 45 variables. Linearization is applied as before to handle nonlinearity. Because triple pendulum is a chaotic system, we apply an alternating-authority MAMPC instead of standard MAMPC with
Evaluation of computational efficiency is also identical to the single pendulum experiment. The system is simulated three times with alternating-authority MAMPC, implicit MPC, and explicit MPC at [π/6 1 π/6 1 π/6 1]. The per-step running times of the three controllers are shown in Figure 6 with the corresponding statistics tabulated in the second column of Table 5. The total running times of the three controllers are summarized in the second column of Table 6. An up time analysis of the three control modes within the MAMPC is tabulated in the second column of Table 7. 6.3. Bicopter. As a parallel robotic application to the pendulum models, the third model is a bicopter as shown in Figure 7. The goal is to hover the bicopter in air.
The state is defined as x := [xẋ yẏ θθ] ∈ R 4 × ([−π, π) × R), where x, y are horizontal and vertical translations and θ is the angle of tilting. The input is defined as u := [u 1 u 2 ] ∈ R 2 , where u 1 , u 2 are the thrust exerted by the left and right propellers, respectively. The equation of motion is described in the following: where definitions and values of parameters are specified in Table 3.  The control objective is to steer the bicopter to the origin by applying limited thrusts u 1 , u 2 ∈ [0. 1, 9.1572] in N at the two propellers. The thrust limits are chosen such that hovering thrusts are approximately in the center of the thrust limit. The MPC design highlights a sampling interval of 0.1 s, a planning horizon N = 20, and eight optimization variables per step, resulting in a LCQP of 2.0 s prediction horizon and 160 variables. To handle plant nonlinearity, we linearize the system around the equilibrium. We apply a standard MAMPC to control the bicopter with The NN is a three-layer, 50-neuron MLP trained through supervised learning with data randomly sampled from x, y, θ ∈ [−π/2, π/2],ẋ,ẏ,θ ∈ [−1, 1].
Evaluation of computational efficiency is identical as before. The system is simulated with the same initial condition at [π/2 1 π/2 1 π/2 1] with standard MAMPC, implicit MPC, and explicit MPC. The per-step running times of the three controllers are shown in Figure 8 with the corresponding statistics tabulated in the third column of Table 5. The total running times of the three controllers are summarized in the third column of Table 6. An up time analysis of the three control modes within the MAMPC is tabulated in the third column of Table 7. 6.4. Quadcopter. As an extension to the ideal bicopter model, the last model is a quadcopter as shown in Figure 9 [1]. The goal is also to hover the copter in air but with more realistic system dynamics.
Evaluation of computational efficiency is identical as before. The system is simulated with the same initial condition at [0.5 0.1 0.5 0.1 0.5 0.1 π/6 0.1 π/6 0.1 π/4 0.1] with way-point MAMPC, implicit MPC, and explicit MPC. The per-step running times of the three controllers are shown in Figure 10 with the corresponding statistics tabulated in the last column of Table 5. The total running times of the three controllers are summarized in the last column of Table 6. An up time analysis of the three control modes within the MAMPC is tabulated in the last column of Table 7.
Note that this experiment is already at the limit of explicit MPC method. The explicit MPC offline caching takes over 10 hours to complete with the resulting explicit control law occupies around 100 MB of hard disk storage. In comparison, NN supervised learning takes less than one hour and only requires less than 20 KB of storage. As explicit MPC method scales exponentially with state dimension, it will not likely work well for problems of higher dimensions.   6.5. Timing Method. The numerical experiments are conducted on a macOS with a 3.2 GHz 6-Core Intel Core i7 processor. All codes are implemented and timed using MATLAB. The running time data in Figure 4 to Figure 10 are measured by repeating and timing the same experiment 50 times and then taking the median of the 50 measurements. Due to initial loading and logging performed by MATLAB MPC package, the first control step of each experiment is very slow and thus omitted from data. The per-step running times are obtained by taking the means and one standard deviations of relevant data along a trajectory. The total running times are obtained by summing the running time of every data point along a trajectory. Up time division is obtained by dividing the running time and the number of control steps each mode of MAMPC takes by the total amount.
We remark that the explicit MPC solver in MATLAB is likely not very optimized. There should be room for improvement at least for the simpler systems like pendulum and bicopter. Nevertheless, we decide to benchmark in MATLAB as it is one of the standard software that the community use for robotic control. We believe that although the exact values of running time in the Section may vary with a different setup of hardware and software, the comparative performance of each control methods should approximately remain the same.
With the numerical experiments presented above, we continue to provide a few key interpretations of the experimental results and controller design in Section 7.  7. Discussions. The numerical experiments above indicates that MAMPC may have superior amortized running time but inferior worst-case running time in common robotic applications. As data shows, the LQR mode and NN mode of MAMPC is faster than implicit and explicit MPC methods. Moreover, we will be able to further reduce the running time of the NN mode by approximately an order of magnitude, if we can safely skip the forward verification and apply the NN directly. However, the MPC mode of MAMPC is slower than the implicit and explicit methods due to overhead from checking switch conditions, leading to increased latency in the worstcase scenario. Despite of the drawback, experiments show that the MPC mode is seldom invoked, which enables MAMPC to have smaller average latency than the implicit and explicit methods.
Beyond the direct implications of the numerical experiments, we make following remarks on set design, forward verification, additional running time optimizations, and limitations. 7.1. Set Design. Identification and design of feasibility set X 0 and attraction set R LQR is very challenging. One one hand, it is easier to just approximate them by conservative set estimates for robustness measures. For example, one can replace X 0 and R LQR with X − 0 and R − LQR , where X − 0 ⊂ X 0 and R − LQR ⊂ R LQR . On the other hand, however, to guarantee computational efficiency, one should make sets R LQR , R NN as large as possible. This way, faster modes of MAMPC get invoked more frequently than the slow, default mode of MAMPC, leading to a more efficient amortized running time performance. Conservative set design makes the system more robust to model uncertainties but also slower in running time. Balancing the trade-off between latency and robustness is therefore a design process.

Forward verification.
The horizon used in forward verification immediately affects the maximum possible running time of the MAMPC. A valid verification horizon should not make the running time of the MPC mode longer than the maximum allowable computational latency. Moreover, an additional rule of thumb is to choose a verification horizon such that the maximum possible running time of NN mode is on par with the running time of the implicit MPC.
The simulation method used for forward verification also directly impacts the worst-case latency of the MAMPC. For continuous plants, we recommend to use forward Euler method with a sampling time that is as large as possible and to use a conservative set design to absorb integration errors. Like designing sets, choosing the right sampling time is a balancing act between latency and robustness.

Additional
Optimizations. For small to moderately sized problems, we may consider developing a fail-free MAMPC as specified in Theorem 5.5, since it will further speed up computation. Fail-free MAMPC prevents controller from ever having to default to fail-safe mode and therefore skips the routine computation of forward verification in NN mode. It significantly improves the amortized and worstcase running time.
If the computational device is capable of executing independent threads in parallel, it may be advantageous to distribute computation across multiple threads to further reduce latency. For example, a possible parallelization strategy for standard MAMPC is as follows: 1) compute the three control modes of MAMPC in three parallel threads, 2) among those that pass their corresponding switch conditions, execute whichever that finishes first, and 3) reset threads for next control step and repeat. This way, the worst-case running time will be roughly identical to that of the original MPC. As a result, the parallelized MAMPC scheme is at worst equivalent to the hybrid strategy in [20].

Limitations.
The foremost limitation of the MAMPC method is that without removing the fail-safe controller it can never reduce the worst-case running time to be below that of the original MPC. This shortcoming limits the applicability of the hyrbid control scheme. For instance, if the original MPC breaks closed-loop stability due to exceedingly high latency, the MAMPC counterpart will also fail to make the closed-loop system stable. In the worst-case scenario, the MAMPC controller is at best equivalent to the original MPC controller. Therefore, MAMPC is only suited for applications where the existing MPC controller works but users demand additional improvement in amortized running time. Examples of such applications include routine repetitive manipulator tasks, where one can acquire proficiency through practice, such as pick-and-place and item assembly.
Another limitation of MAMPC is that the MPC designs presented in the four numercial experiments are neither optimal nor certified. For example, instead of linearization around the equilibrium, nonlinearity may be handled directly by a nonlinear solver. Also, no verification has been done to check that the MPC controllers obtained through plant linearization are persistent feasible and closed-loop stable with respect to the original nonlinear plants. Hence, the controllers in the numerical experiments are not solutions for deployment. Instead, they are academic prototypes designed specifically to investigate the computational aspects of the proposed method.
As the final remark, we conclude our work in Section 8.

Conclusion.
To improve average computational efficiency of MPC, we have developed a triple-mode hybrid control named MAMPC. Stability of MAMPC is guaranteed via forward simulation, while efficiency is achieved by replacing MPC with a more efficient NN or LQR, whenever stability permits. Results indicate that MAMPC often has a better amortized running time but a slightly prolonged worstcase running time.
As future work, one may generalize the hybrid control pattern in MAMPC to other controllers beyond MPC. The hybrid control pattern consists of three basic ingredients: an efficient equilibrium policy, a frugal heuristic policy, and an expensive default policy. When the state is close to its equilibrium, we may effectively approximate the system dynamics with a linear system and control it with a linear controller. Developing such linear controller can be achieved through LQR or other linear control methods in either frequency or time domain. Away from the equilibrium, first-order approximation will no longer be effective, so a new controller is required to maintain system stability. In this case, one can attempt to leverage machine learning methods, such as supervised learning or reinforcement learning, to train a heuristic controller that is empirically effective and efficient. Lastly, when the heuristic policy is expected to fail, we should invoke a default policy to maintain the system in control, or at least to prevent the system from catastrophic failures. In this work, the default policy is MPC but this can be other nonlinear controllers too.