{'fraction_non_alphanumeric': 0.05446775446775447, 'fraction_numerical': 0.011188811188811189, 'mean_word_length': 4.22362012987013, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Revisiting the derivation of stage costs in infinite horizon discrete-time optimal control

In many applications of optimal control, the stage cost is not fixed, but rather a design choice with considerable impact on the control performance. In infinite horizon optimal control, the choice of stage cost is often restricted by the requirement of uniform cost controllability, which is nontrivial to satisfy. Here we revisit a previously proposed constructive technique for stage cost design. We generalize its setting, weaken the required assumptions and add additional flexibility. Furthermore, we show that the required assumptions essentially cannot be weakened anymore. By providing improved design options for stage costs, this work contributes to expanding the applicability of optimization-based control methodologies, in particular, model predictive control.


I. INTRODUCTION
Infinite horizon optimal control problems are at the core of many modern control approaches. For example, model predictive control (MPC) can be interpreted as a finite horizon approximation to a given infinite horizon optimal control problem [3, Section 5.4], [2]. An important component of an optimal control problem is the stage cost, the instantaneous cost of an applied control input and the current state of the system. In many control applications, the stage cost is not completely specified, but rather a design parameter. Examples include automatic controller tuning [6] and MPC without terminal conditions [3,Chapter 6]. Especially in the latter case, goals like stabilization and constraint satisfaction are primary, and the concrete choice of a stage cost is secondary. The major technical challenge is then the design of a stage cost leading to a uniformly bounded value function of the corresponding infinite horizon optimal control problem, i.e., uniform cost controllability, cf. Section III for details.
A constructive method is described in [3,Section 4.1] that allows to derive a stage cost from certain uniform controllability assumptions, leading to uniform cost controllability. Due to the relevance of good stage cost designs, we revisit this technique in the present work. In particular, we use a more general setup and weaken the necessary controllability assumptions. Furthermore, we allow for more flexibility in the design of the stage cost and cover explicitly the case of interaction terms between state and control, which seems to have not been discussed so far. Finally, by providing a converse result, we show that the controllability assumptions essentially cannot be further weakened in general, an observation that has not been explicitly made before, to the best of our knowledge. Outline In Section II, we introduce the problem setting and provide some necessary technical background. The relevant controllability assumptions are introduced and investigated in Section III and then used in Section IV to derive suitable stage costs. We demonstrate in Section V that the utilized controllability assumptions are indeed reasonable in this context. Section VI concludes with some discussion and directions for future work.

II. PRELIMINARIES
We consider discrete-time nonlinear control systems described by a transition function f : X × U → X. The state space X and the set of control inputs U can be arbitrary nonempty sets. As usual, for a given input sequence u, the resulting trajectory is defined by φ(0; x, u) = x φ(n + 1; x, u) = f (φ(n; x, u), u(n)), n ≥ 0.
Similar to for example [1] or [10], we work in a very general setting using a state measure σ : X → R ≥0 and an input or control measure ρ : U → R ≥0 . For the following developments, we do not require any special properties of σ and ρ.
Example 1: This general setup covers most relevant settings, including the following cases: 1) (X, d X ) a metric space, A ⊆ X a nonempty, compact subset, and σ(x) = inf This is essentially the (time-invariant version of the) setting in [3,Section 4.1].
3) Probably the most common case is X = R n , U = R m , (0, 0) ∈ R n × R m an equilibrium and For a stage cost ℓ : X × U → R ≥0 and horizon length N ∈ N + ∪ {∞}, define the total cost for initial state x ∈ X and input sequence u ∈ U N (setting U ∞ = U N ) by ℓ(φ(n; x, u), u(n)). (2) Finally, we use extensively the popular comparison function formalism. The class K consists of all functions α : R ≥0 → R ≥0 with α(0) = 0 and that are continuous and strictly increasing. The class K ∞ consists of functions α ∈ K such that lim r→∞ α(r) = ∞ and the class KL consists 1 of continuous functions β : R ≥0 × R ≥0 → R ≥0 such that for all t ∈ R ≥0 we have β(·, t) ∈ K ∞ and for all r ∈ R >0 the function β(r, ·) is strictly decreasing and lim t→∞ β(r, t) = 0. To improve readibility, we often apply pointwise unary and binary operations as well as relations to comparison functions without making the argument explicit. For example, for α 1 , α 2 ∈ K ∞ , the expression α 1 ≤ α 2 means ∀r ∈ R ≥0 : α 1 (r) ≤ α 2 (r).
The next lemma collects some properties of comparison functions that are needed in the remainder of this work. For proofs and further background, see for example [4].

III. ASYMPTOTIC AND COST CONTROLLABILITY CONDITIONS
We now summarize the controllability notions used in the sequel. Note that we use some slightly nonstandard, but descriptive terms.
Definition 1: Let X ⊆ X. We say that system (1) is 1) uniformly asymptotically controllable (UAC) on X if there exists β ∈ KL such that for all 2) uniformly asymptotically controllable with uniformly vanishing controls (UACwUVC) on X if there exist β x , β u ∈ KL such that for all x ∈ X there exists 3) uniformly asymptotically controllable with uniformly bounded generalized energy controls (UACwUBgEC) For convenience, denote the set of all such control sequences by U ∞ UBgE (x). Remark 1: If the system is UACwUVC, then for all ). This shows that in the setting of Example 1, item 2), the definition of UAC and UACwUVC reduces to uniform controllability with the small control property as described in [3,Definition 4.1].
Remark 2: The definition of UACwUBgEC is motivated by the case η(r) = r 2 which can be interpreted as physical energy. Note that we focus on η ∈ K ∞ since this aligns closest with the interpretation of generalized energy. Conceptually, our definition is similar to summability properties, cf. e.g. [8], but we are not aware of previous usages in the context of asymptotic controllability. Next, we clarify some relations between the different concepts from Definition 1.
Proposition 1: 1) If system (1) is UACwUVC on X, then it is also UACwUBgEC on X. 2) If system (1) is UACwUBgEC on X, then it is UAC.
In particular, Proposition 1 shows that the UACwUVC property is at least as strong as UACwUBgEC.
Remark 3: We almost have a converse of the first implication in the preceding proposition. If the system is UACwUBgEC, then ∞ n=0 η ρ(u(n)) ≤ γ(σ(x)) < ∞ and η ∈ K ∞ imply that ρ(u(n)) → 0 for n → ∞, but the convergence rate might depend on x and u x , and hence we might not have uniformity. We conjecture that under mild compactness and continuity assumptions we can ensure this uniformity and hence UACwUBgEC and UACwUVC become equivalent.
Definition 2: We say that system (1) is uniformly cost controllable (UCC) on X ⊆ X with respect to (w.r.t.) the stage cost ℓ : The UCC property is sometimes referred to as weak controllability, see for example [7, Assumption 2.17].

IV. DERIVING STAGE COSTS FROM ASYMPTOTIC CONTROLLABILITY
The starting point is the derivation of stage costs of the form ℓ(x, u) = q(σ(x)) + r(ρ(u)) for q ∈ K ∞ and some r : R ≥0 → R ≥0 . The following result is a direct adaption of [3,Theorem 4.3] to the present setting, allowing for additional flexibility in the choice of q and r and requiring only UACwUBgEC instead of UACwUVC.
More explicitly, for any KL-decomposition β(r, n) ≤ Proof: Let x ∈ X be arbitrary and choose a corresponding u x ∈ U ∞ . For any n ≥ 0 we then have q σ(φ(n; x, u x )) ≤ q β(σ(x), n) ≤ q γ 2 θ n γ 1 (σ(x)) ≤ C q θ n γ 1 (σ(x)), where we used the monotonicity of q together with the KLdecomposition of β and the choice q ≤ C q · γ −1 2 . For N ≥ 1 we can continue with where we used additionally the choice r ≤ C r · η and (7). Since the last term is independent of N (together with nonnegativity of ℓ) we get convergence of the total cost together with the bound The remaining claims are clear.

Remark 4:
The existence of α ∈ K ∞ with ℓ(x, u) ≥ α(σ(x)) for all (x, u) is crucial for stabilization tasks, cf. [3,Chapter 4]. Since q ∈ K ∞ , this is trivially satisfied here (actually, q could be replaced by anyq ≤ q withq ≥ α for some α ∈ K ∞ ). We now investigate which interaction terms between x and u might be possible while still ensuring UCC. The following simple result already allows a considerable amount of flexibility.

Continuing with
where we used the KL-decomposition from Theorem 1, and where we used again (7), leads to s(φ(n; x, u x ), u x (n)) ≤ C 1 θ n γ 1 (σ(x)) + C 2 η(ρ(n)) Summing over n yields and the claim follows by combining this with the results from Theorem 1. Example 2: Consider the situation of Theorem 1 for the case of UACwUBgEC. The goal is to find functions s compatible with Proposition 2.
By construction, the set I 1 has at most N (σ(x)) elements and we have the partition For n ∈ I 1 we find (since σ(n) ≥ R σ ) Since σ(n) < R σ for all n ∈ I 2 , we get from the proof of =:α 2 (σ(x)).
Using standard arguments we can now upper boundα 1 +α 2 by a functionᾱ ∈ K ∞ . Remark 5: 1) The bound (10) is only relevant for states with large state measure σ, corresponding to the transient behavior in the context of stabilization. Therefore, a rough bound of this form is enough. In particular, using the additive form considered here does not lead to a loss in generality. For example, assume we have a bound ℓ(x, u) ≤ B(σ(x), ρ(u)) in (10) instead, where B : R ≥0 × R ≥0 → R ≥0 is continuous and nondecreasing in the natural partial order on R 2 ≥0 (i.e., ). Using arguments similar to [4, Lemma 33], one can then use a bound of the form (10) again.
2) In the bound (10), the behaviour of α x | [0,Rσ ) does not matter. In particular, this justifies using only the term α u (ρ(u)) to include penalization of ρ(u). For example, if we want to penalize ρ(u) = 0 if σ(x) ≥ R σ , then this can be included in the term α x .

V. A CONVERSE RESULT
In order to allow flexibility in the design of stage costs, it is desirable that the techniques described in the previous section yield a large range of stage costs. In particular, the question arises whether the controllability conditions used to derive stage costs are much stronger than UCC with respect to a stage cost of a certain type. Since under mild assumptions a stage cost as constructed in Theorem 1 can be used to derive a stabilizing controller, cf. [3,Section 4.3], it is clear that UCC implies some form of controllability. The next result provides, under an additional controlledinvariance assumption, a converse to Theorem 1.
Proof of claim: Let m ≥ 1 and n ≥ M m be arbitrary. Let m * be the largest positive integer such that n ≥ M m * , then σ(φ(n; x, u)) = σ(φ(n − M m * ; φ m * , u m * )) where we used ℓ ≥ 0 and the choice of u m * in the first inequality, property (i) of u m * established in Step 2 for the second inequality, the inductive definition of φ m * for the third inequality, the definition ofǫ m * in the fourth inequality and finally m * ≥ m. Altogether, this shows (i). Next,
Corollary 1: System (1) is UACwUBgEC on X if and only if it is UCC on X with respect to a stage cost ℓ(x, u) = q(σ(x)) + r(ρ(u)) with q, r ∈ K ∞ . An interesting feature of the proof of Theorem 3 and hence of Corollary 1 is that it is elementary (no auxiliary tools from Dynamic Programming or Lyapunov theory are used) and constructive (at least in principle explicit formulas are derived) nature.

VI. CONCLUSION
In this work, we generalized the constructive method described in [3, Section 4.1] for finding a stage cost for infinite horizon optimal control ensuring uniform cost controllability, and we investigated the required controllability conditions. The generalization to the case of time-varying systems with corresponding uniformity assumptions is straightforward and has been omited due to space constraints. Furthermore, at least for summable nonnegative weighting sequences, the case of a total cost functional with weighting is also a straightforward extension of our results. Interesting directions for further investigations include the development of more explicit techniques for manipulating comparison functions and corresponding numerical algorithms. Another interesting direction is the characterization of all stage costs with certain properties like prescribed growth rates or convexity, under given controllability assumptions.