{'fraction_non_alphanumeric': 0.03157384987893463, 'fraction_numerical': 0.012784503631961259, 'mean_word_length': 4.855117663736887, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Robust Optimal Control-based Design of Combined Chemo- and Immunotherapy Delivery Profiles

Abstract This paper addresses the problem of drug injection schedules design for cancer treatment, in the presence of model parametric uncertainties. It is commonly known that achieving optimal recovery performances under uncertainties is a complex task. Therefore, we propose to use a recent optimal control approach, based on the moment optimization framework. This method allows to formulate and solve robust optimal control problems by taking into account uncertain parameters and initial states, modeled as probability distributions. We analyse a two dimensional model that describes the interaction dynamics between tumor and immune cells. Furthermore, we derive statistically optimal combined strategies of chemo-and immunotherapy treatments, assuming the knowledge of probability distributions of some uncertain model parameters, namely, the tumor growth rate and the rate of immune cells influx. Numerical simulations are carried out in order to illustrate the effects of parametric uncertainties on dynamics, when using a nominal injection profile. Finally, we compare the recovery performance of nominal and robust schedules.


INTRODUCTION
Control design for biological systems is a very promising topic in the sense that robust control theory can help to handle the uncertainties which are due to the system dynamics complexity, the different neglected phenomena and the lack of knowledge on model parameters.
It is well known that biological phenomena, in particular tumors, are highly uncertain systems. Both evolution and treatment effects are patient-dependent. In the context of cancer treatment, control theory can help to design new drug delivery schedules and to combine effectively many therapies, in the presence of parametric uncertainties, in order to achieve better treatment performances.
The advances in genetics that have taken place recently led to considerable progress in experimental and clinical immunology (Eftimie et al., 2016) and many researches on modeling the immune system dynamics had been carried out. Since the mathematical modeling of the entire immune system can be a very complex task, researchers focus on the elements of the immune system that are known to be significant in controlling the tumor growth (De Pillis, 2006). The readers interested in tumor-immune interactions modeling can refer to De Pillis et al. (2007),De Pillis et al. (2009), d 'Onofrio et al. (2012), Eftimie et al. (2016) and references therein. The model considered in this paper describes the interactions between two populations, tumor cells and immune effector cells. It also includes explicitly two therapies delivery, cytotoxic chemotherapy and immunostimulation. This model takes into account the chemotherapy-induced loss of tumor cells and incorporates the benefical effects of the immune system on controlling the tumor growth.
Cancer treatment usually implies handling state and input (drugs amounts) constraints, some nonlinearities, many uncertainties and optimality issues. It is definitely the collection of all complexity ingredients in terms of control design. We can find in the literature many works regarding the application of optimal control methods on cancer treatment problems. For instance, Ledzewicz and Schättler (2008), Schättler et al. (2011) andCerrai (2009), where optimal protocols for anti-angiogenic treatment were investigated, or De Pillis et al. (2007) who designed quadratic and linear controls for a tumorimmune interactions model with chemotherapy delivery. However, robust control is not yet extensively studied for cancer treatment. We can cite for example, Alamir (2014), where a robust feedback scheme is proposed to schedule antiangiogenic treatment combined with chemotherapy. Kovacs et al. (2014) applied an H ∞ based robust control to the same model.
In this paper, we propose to formulate robust optimal control problems in the moment optimization framework by lifting a polynomial nonlinear dynamical system to the infinite-dimensional space of measures. Since the problem is reformulated in the space of probability measures, it is straightforward to define states and parameters as random variables characterized by their probability distributions.
The resulting infinite-dimensional problem is solved using truncations known as finite-dimensional semidefinite (SD) hierarchies (Lasserre, 2001), providing a converging sequence of lower bounds on the optimal solution.
In Section 2, we present the dynamical model used for numerical simulations. Section 3 recalls the main key points of the generalized moment problem for optimal control. The optimal control problems to be solved and the numerical results are presented in Section 4. Finally, in Section 5, we discuss advantages and limitations of this approach and we set some perspectives for future works.

DYNAMICAL MODEL
In this paper, we will consider a modified version of the two dimensional model presented in d' Onofrio et al. (2012), which describes the interaction dynamics between the tumor and the immune system. This model had been intensively used in the literature in order to investigate its equilibria and propose some optimal control strategies. For instance, Sharifi et al. (2017) proposed a multiple model predictive control scheme to design chemo-and immunotherapy injection schedules.
We propose to replace the Gompertzian tumor growth model in d 'Onofrio et al. (2012) with a logistic one , which leads to the following polynomial dynamics : where x 1 and x 2 denote, respectively, the number of tumor cells and the density of effector immune cells (ECs), u 1 and u 2 are respectively, the delivery profiles of a cytotoxic agent and an immunostimulator. Model (1) has two locally asymptotically stable equilibria. The macroscopic malignant equilibrium is (x m , y m ) (735.9, 0.032) and the benign one is (x b , y b ) (34.98, 0.53).
It is important to notice that the treatment performance depends highly on the initial conditions, since there is a coexistence of macro-and microscopic equilibria.

OVERVIEW ON GENERALIZED MOMENT PROBLEM FOR OPTIMAL CONTROL
In this section we will provide an overview on the basic concepts and computational tools for solving optimal control problems by resorting to the generalized moment problem approach. This method, developed by Lasserre (Lasserre, 2000), is based on the fact that polynomial optimization problems (thus, a class of nonconvex finite dimensional problems), are equivalent, under mild assumptions, to linear problems in the space of measure, that are infinite dimensional. Although the infinite dimensional nature makes the latter problems hardly manageable, relaxations can be obtained, providing a converging sequence of lower bounds on the global optima, under some compactness assumptions. Therefore, generating and solving such sequence of relaxations, referred to as Lasserre hierarchy (Lasserre, 2001), allows to approach the exact solution of the original polynomial optimization problem with arbitrary precision.
This approach has been recently extended to optimal control problems with a polynomial structure and bounded constraints (Lasserre et al., 2008), for which sequences of convex problems can be determined to obtain suboptimal solutions converging to the exact optimal control. Moreover, as the linear infinite dimensional problems are defined over the space of measures, which can be naturally employed to model probability distributions, this approach allows to address and solve optimal control problems involving probability distributions as states and parameters.
Consider the polynomial optimal control problem where x ∈ R n is the state, u ∈ R m is the input, functions f : R × R n × R m → R, L : R × R n × R m → R and Φ : R n → R are polynomials and the constraints sets X 0 , X, X T and U are compact basic semi-algebraic sets, i.e. defined as the union of finitely many intersections of closed polynomial superlevel sets. Under some additional assumption on the convexity of the set f (t, x, U ) for all t and x, the optimum of (2) is the same as the optimum of a linear infinite dimensional optimization problem defined over the space of probability measures, given below.
For this, we briefly introduce the concept of measure and moments, the reader interested in a more detailed formalization is referred to the related literature, e.g. Lasserre (2010) and Lasserre et al. (2008). Given a compact set X ∈ R n and the Borel σ-algebra B(X), that is a particular set of subsets of X containing all the open subsets of X, a Borel measure µ on X is a function that associates a real value to every element of B(X). A measure is nonnegative if it takes only nonnegative values, is a probability measure if it is nonnegative and µ(X) = 1. An example of a positive measure is the Lebesgue measure, that associates the surface to sets in R 2 , the volume to sets in R 3 etc., while the Dirac measure δ z (x) with z ∈ X is a probability measure that assigns the value 1 to every subset of X containing z and 0 otherwise. The spaces of measures and positive measures are denoted M(X) and M + (X), respectively. Measures can also be defined as the space of linear functionals acting on the space of functions that are continuous on X, i.e. by the action they have over the elements of the dual space for all v ∈ C(X). The following linear problem in the space of measures inf is infinite dimensional, has uncountable many constraints and has the same optimum value as the original optimal control problem (2), under mild assumptions, see Lasserre et al. (2008).
Such a problem remains highly complex and unmanageably large, however, Lasserre hierarchy (Lasserre, 2001) of relaxed LMI problems can be determined to obtain suboptimal solutions that converge, under certain compactness and convexity assumptions, to the optimal solution of the original optimal control problem. To obtain the relaxations, one has first to consider the relation between the measure µ 0 , µ and µ T and their moments. Given x ∈ R n and σ ∈ N n , the moment of order σ ∈ N n of µ ∈ M(X) is defined as : where x σ = n k=1 x σ k k . LMI conditions can be given in terms of the moments of µ that are equivalent to the constraint µ ∈ M + (X), conditions that still involve infinite dimensional matrices which are functions of the infinitely many variables y σ for all σ ∈ N n . The relaxations consist, in practice, in considering the matrix structures obtained by appropriately truncating the vector of moments to a finite maximal degree (r) and imposing in (3) constraints over polynomials of a finite maximal degree in spite of all v ∈ C 1 ([0, T ] × X). This leads to a hierarchy of finitedimensional semidefinite programming problems whose solutions converge to the solution of the optimal control problem as the relaxation degree grows.
The particularly interesting feature of this approach is the fact that, even in case of deterministic dynamical systems, the initial state as well as the final one and the state along trajectories, are dealt with by defining measures on the state space, see (3). The same holds for the input. For instance, if x 0 = x(0) ∈ X 0 is a singleton, then the initial measure µ 0 in (3) should be imposed by fixing, for all σ ∈ N n , its moments given as : Therefore, the polynomial optimization method based on measures is particularly suitable for dealing with states and inputs that are characterized by probability distributions, by simply imposing the moments of the related probability density functions.
In the particular case under study, we aim at designing a robust optimal control for a dynamical model describing the tumor growth, the parameters of which are supposed to be not perfectly known. This lack of knowledge can be modeled through uncertain parameters characterized by probability distributions, with compact support. Then, in practice, it is sufficient to define an extended state containing both tumor and immune cell populations and the uncertain parameters, i.e. µ C and α, see (1), and to impose their time invariant characteristic through the dynamicsμ C (t) = 0 andα(t) = 0. Thus, supposing that η(µ c ) and ν(α) denote the probability distributions of parameters µ C and α, the optimal control problem (3) to be solved should have as initial condition , imposed through moments of the initial measure.

Nominal optimal control problem
Similarly to d 'Onofrio et al. (2012) and Sharifi et al. (2017), we assume that the initial state of the system dynamics (1) is (x 10 , x 20 ) = (600, 0.1), we also consider that the maximum drug dose is 1 for both chemotherapy and immunotherapy. Furthermore, we add constraints on the immune cells density and the number of tumor cells in order to ensure the compactness of the state set X. Another constraint on the final tumor size is imposed in order to drive the tumor to the benign region. The nominal (i.e. considering nominal values of model parameters) optimal control problem that we propose to solve for t ∈ [0, 60] is the following: min u1(·),u2(·) x 1 (0) = 600, x 2 (0) = 0.1, The cost J is chosen according to the objectives that one seeks to achieve. It can contain many terms such as final states, integrals of state trajectories and control inputs, with different penalties in order to achieve a trade-off between the different control objectives. Problem (5) can be reformulated in the framework of moment optimization via GloptiPoly 3 (Henrion et al., 2009), as explained in Section 3, and can be solved using YALMIP and the semidefinite programming solver MOSEK. The control inputs are approximated, based on the knowledge of their moments, using Christoffel-Darboux kernel, for more details, see (Marx et al., 2019). For practical reasons, time and states trajectories are scaled to [0, 1], therefore, the control inputs presented in this paper are computed for scaled dynamics.
Let's consider for instance the minimization of the cost J 1 = x 1 (60). Figure 1 shows the approximations of control inputs that we obtained after solving the reformulated problem corresponding to (5) with J = J 1 . The evolution of state trajectories with those controls is presented in Figure 2, we can see that the tumor burden decreases slowly to reach the final value that lies in the benign region. Assuming that we want the tumor size to decrease faster, we can minimize J 2 = x 1 (t)dt. The approximated control inputs are presented in Figure 3, we can notice that the chemotherapy profile is aggressive and persistent, this is due to the choice of the cost which considers only the minimization of the integral of x(t). Such controls are not allowed practically because of the high toxicity of the cytotoxic agent.  Figure 4 shows that the state corresponding to the tumor cells, x 1 , goes to 0 faster than in Figure 2. Furthermore, we can see that the immune cells density goes up rapidly to reach very high values (thus, violating the imposed constraint 0 x 2 5).
Since chemotherapy has damaging side effects on the human body, it is common to frame an optimal control problem so that the total amount of drugs is minimized (De Pillis et al., 2007). It is also important to take a look at the evolution of the immune system, because the immune-weakening has damaging effects on the human body. Thereby, one can easily notice that the choice of the cost J, to be minimized, is very important in order to meet the control objectives. Now, we propose to minimize J 3 = x 1 (60)+0.4 x 2 (t)dt+ 0.01 u 1 (t)dt + 0.01 u 2 (t)dt. As we can see in Figure 5, penalizing the control inputs integrals allows to reduce considerably the injected drugs amounts. In Figure 5, we show the graphs in the time interval [0, 5] to emphasis the differences between the two profiles, since for t ∈ [5, 60], u 1 (t) = 0 and u 2 (t) = 0.   Figure 6 shows that the states converge to the benign equilibrium at around 30 days. Therefore, the control profiles approximated by minimizing J 3 allow to satisfy perfectly the standard control objectives, since they drive the states to the benign equilibrium (x b , y b ). Furthermore, we can notice that those drug injection profiles minimize rapidly the tumor while maintaining a relatively strong immune system. Even though controls in Figure 5 satisfy standard objectives in the context of nominal optimal control, we will show that when the dynamics are subject to parameters uncertainties, those controls will not meet the goals set in the optimal control problem.
Remark 1: The considered distributions are not based on practical knowledge of the system parameters, they are chosen only to illustrate the problem of handling parametric uncertainties. The robust schedules will be designed considering truncated distributions in order to satisfy compactness conditions. Figure 7 presents 100 Monte-Carlo simulations with random values of µ C and α (the random selection is carried out according to their corresponding probability distributions). It shows that there is a probability of 19% for the states to converge to the malignant equilibrium (x m , y m ) (i.e. leading to patients death). Therefore, it is crucial to consider the potential uncertainties on model parameters.

Robust optimal control problem
Let's extend system (1) to the following dynamics: The state augmentation in (6) allows to characterize µ C and α by their probability distributions, as explained in Section 3. Similarly to problem (5), one can reformulate the robust optimal control problem with dynamics (6) by including the moments of the parameters distributions. Figure 8 presents a comparison between nominal and robust injection schedules, approximated after minimizing the cost J 3 in the nominal case and E (J 3 ) in the robust case, since we have a flow of trajectories, generated by the parameters distributions . We can notice that similarly to the nominal profiles, the robust ones are also single doses injected at the begining of the treatment. However, we can see that the robust profiles use more amounts of drugs which highlights the importance of taking into account parametric uncertainties description.  Figure 9 presents 100 Monte-Carlo simulations using the approximated robust injection profiles. We can notice that the probability of convergence to the malignant equilibrium has been reduced from 19%, using the nominal profiles, to 8%, in the case of robust schedules.
Using the asymptotic equivalence between (7) and (8), we computed the costs corresponding to nominal and robust profiles, based on Monte-Carlo simulations that we carried out for both schedules. Table 2 shows that the mean and variance of the costs corresponding to robust schedules, are considerably less than those of the nominal costs. This is mainly due to the excessive number of constraints violations that occur when applying nominal controls.  Table 3 presents a comparison between the computational times of the nominal optimal control problem (OCP) and the robust one. We notice a considerable difference in the computational cost, it is mainly due to the increase of the problem dimension, after performing dynamics extension to solve the robust OCP. Increasing the relaxation order r allows to have better approximations on the moments, however, it increases the problem dimension and therefore, the computational time.

CONCLUSION AND DISCUSSION
We presented in this paper preliminary results on the application of moment optimization theory to schedule cancer treatment. We highlighted the importance of taking into account parametric uncertainties in the optimal control problem. Furthermore, we designed robust and optimal combined chemo-and immunotherapy injection profiles that allow to meet specific objectives.
The moment optimization approach can be very promising for many applications, since it allows to reconstruct optimal injection schedules for a class of nonlinear systems with parametric uncertainties considerations. However, it may have some limitations, mainly the restriction on polynomial dynamics and the limited dimension (state and control variables) that can be handled. Although the required computational time is high in the case of solving a robust OCP, in some applications, it remains crucial to guarantee robust performances.
Finally, furture work will be focused on exploring the uncertainties effects of other model parameters on cancer schedules treatment design, including uncertainties on initial states and investigating the consequences of adding a minimal immune cells density constraint in the OCP.