{'fraction_non_alphanumeric': 0.04429158212500619, 'fraction_numerical': 0.019894096105310042, 'mean_word_length': 4.641541038525963, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
On the dynamics and robustness of the chemostat with multiplicative noise

The stochastic dynamics of a two-state bioreactor model with random feed flow fluctuations and non-monotonic specific growth rate is analyzed. Using the Fokker-Planck equation approach for describing the probability density function (PDF) evolution the lack of stochastic robustness due to deterministic bifurcation phenomena for the open-loop reactor operating under optimal (maximum production) operation condition is established, and the associated stochastic stabilization problem is addressed. Inherent differences between the presence of multiplicative noise, due to the feed flow fluctuations, and additive background noise are analytically established. Numerical simulation results illustrate these inherent differences, the stochastic fragility of the open-loop operation yielding a stochastic extinction phenomenon, as well as the stochastic PDF stabilization with a proportional feedback control.


INTRODUCTION
Stochastic fluctuations are present in different levels of magnitude in all technical processes [Åström 1970, Jazwinski 1970, Risken and Frank 1996, Krstic and Deng 1998]. Their influence is often neglected, or sometimes approximated with local Gaussian distributions [Åström 1970]. The nonlocal solution behavior of the associated stochastic differential equation (SDE) can be analyzed using the associated Fokker-Planck equation (FPE) [Risken andFrank 1996, Horsthemke andLefever 1984], with the possibility of drawing analytic solutions for the stationary state probility density function (PDF) and for the multi-(deterministic, diffusion and escape)time scale of the transient PDF of single and two-state systems. This approach is employed in the present study for the analysis of an open-loop (OL) and closed-loop (CL) two-state bioreactor with multiplicative noise in the feed flow rate.
In most of the recent stochastic bioreactor studies, Monte Carlo based simulation analysis has been employed [Chen and Zhang 2013, Zhang et al. 2014, Meng et al. 2016, Wang et al. 2017, Sun et al. 2017] lacking the possibility of assessing steady-state multimodal PDFs and PDF transients along diffusion and escape time scales [Risken andFrank 1996, Alvarez et al. 2018]. Studies focussing on the solution behavior of the associated FPE are much less frequent. Within these, most employ a direct numerical solution, like in [Campillo et al. 2014, Voulgarelis et al. 2018]. An analytic analysis of the PDF solution behavior has been performed in a pioneering study [Stephanopoulos et al. 1979] with the single-state approximation of a three-state bioreactor with Monod kinetics, exploiting characteristic deterministic time scales, but overlooking diffusion and metastability scales. Accordingly, the potential of the FP PDE-based theory [Risken andFrank 1996, Alvarez et al. 2018] for the analytic assessment of the PDF behavior in terms of nonlinear deterministic dynamics and stochastic (diffusion and escape) time scales has not been further exploited for the analysis of OL and CL bioreactors.
In recent works on chemical continuous stirred tank reactors [Tronci et al. 2011, Baratti et al. 2016] this potential has been exploited yielding important insights in the PDF OL and CL solution behavior. Besides the fundamental understanding of the complex and sometimes counterintuitive interconnections and implications of stochastic fluctuations on the reactor dynamics, important insights into aspects of modeling, robustness and robust feedback control have been obtained. In particular it has turned out that the inherent differences between additive and multiplicative noise [Horsthemke and Lefever 1984, Risken and Frank 1996, Baratti et al. 2016 with respect to the associated PDF shape and dynamics imply either minor or a substantial difference between the mean and the mode of the associated non-gaussion PDFs, depending on the specific system and its operation condition. The implications of this subtlety on the design of feedback controllers has been exemplarily discussed, e.g., in ]. The stochastic robustness properties and their importance in reactor operation close to deterministically structurally unstable [Andronov and Pontryagin 1937] steady states have been delimited on noise-dependent stochastic time scales for the transition between safety critical reactor operation regimes. This global nonlinear dynamics based FPE approach is applied in the present paper for accessing the fundamental PDF solution behavior of a two-state bioreactor operated in deterministically optimal (maximum production) mode with stochastic fluctuations in the feed flow rate, implying a multiplicative noise excitation. Analytic and numerical solution aspects of the associated FPE are discussed as well as the impact on the most probable state behavior of the CL reactor with proportional feedback control, extending the previous results in [ Baratti et al. 2018] to the case of a two-state reactor with non-isotonic Haldane kinetics.

PROBLEM STATEMENT
Consider the dimensionless model of a bioreactor in chemostat operation ds denotes the concentration in g/l, yield coefficient Y and substrate feed concentration c se in g/l, dimensionless substrate concentration s(t) = c s (t)/(c se ) ≥ 0, dilution rate θ = q/V with flow rate q(t) ≥ 0 and constant volume V > 0, growth rate constant k 0 , and non-monotonic specific growth rate In the sequel it is considered that the flow rate q is subject to high-frequency fluctuations, i.e. q(t) =q +q(t) + w q (4) with constant nominal valueq, controlq, and stochastic fluctuation w q that can be modeled as a white noise, i.e. w q (t) ∼ N (0, σ 2 q ) (5) with associated zero mean normal distribution with standard deviation σ q .
Introducing the state x(t) = [s(t) m(t)] at time t ≥ 0, the stochastic dynamics can be compactly written aṡ Following ], the stochastic differential (called Langevin) equation (7) is written aṡ Note that for all piecewise continuous, bounded input functions u : [0, ∞) → R + the deterministic system with w = 0 and initial condition given that (see, e.g., Waltman 1995, Schaum et al. 2012]) for the deterministic bioreactor model with either monotonic and non-monotonic growth rate µ (i) no limit cycle solutions exist, (ii) there exist between two (for monotonic growth rates) or three (for non-monotonic growth rates) steady statesx i , i ∈ N, (iii) for m + > m in and s + ≥ s in the set X is positively invariant 1 and thus, (iv) in virtue of the theorem of Poincaré-Bendixson all trajectories converge to steady state solutions.
Given the stochastic setup of (7d) with white noise inputs and initial state distribution π 0 , the state is described in a stochastic way using the PDF π : [0, ∞) × X e → R with for all t ≥ 0. From the application of Stratonovich's stochastic calculus [Risken andFrank 1996, Jazwinski 1970] to the SDE (7d) it follows that the state PDF π is the unique solution of the FPE with drift terms and boundary conditions The extended state space X e is introduced in order to accommodate well the state excursions due to noise injection, considering that the probability of a solution of the SDE (7d) to stay in X e is the same as the one for the deterministic solutions to stay in X .
Denote by τ π the solution of (9) so that for any t ≥ 0 it holds that π(t, ·) = τ π (t, ·; π e0 , u). In particular, for a constant u it holds true that π asymptotically reaches a stationary PDFπ along probability convection (t x ) and diffusion (t d ) time scales, as well as along escape (t e ) time scale when metastability (MS) is at play [Horsthemke and Lefever 1984, Risken and Frank 1996, Tronci et al. 2011, i.e., In the sequel, the qualitative dependency of the PDF π on the Damköhler number δ is investigated for the optimal (maximum conversion) operation by following the approach in [Tronci et al. 2011, Baratti et al. 2016, i.e., delimiting the stochastic stationary behavior along the deterministic bifurcation behavior in the understanding that the onset of MS goes at hand with deterministic bifurcation phenomena. These considerations are rounded up by some conclusions about the CL behavior for u = 0.

OPTIMAL REACTOR OPERATION
In this section important facts about the reactor behavior are summarized that will be put in perspective for the stochastic analysis. For this purpose, note that the maxima (i.e., modes) of the PDF (9b), (9c) correspond to the zeros of the drift terms ϕ i , i = 1, 2 which for q θ = 0 and constant flow input u coincide with the deterministic steady-state solutions. Accordingly, the next considerations are directly carried out for the stochastic setup (7d).

Bifurcation behavior
The deterministic steady state solutionsx i = [x 1,i , s e ] , i = 1, 2(, 3) for u = 0 are determined by the algebraic equation 0 = ϕ 1 (x 1,i , s e , 0), q θ = 0. (10) It is known from previous studies (e.g., [Schaum et al. 2012]) that this equation has between one and three solutions depending on the parameter triplet (δ, k s , k i ). For q θ = 0 the steady-state condition (10) modifies to Geometrically, this can be interpreted as the intersection points of three curves in the space J = [0, s + ] × [0, γ + ] with s + ≥ s e and γ + ≥ max{µ (see also [Schaum et al. 2012]), there occurs a saddlenode bifurcation at s × = √ k s k i when (1 + q θ )/δ = µ(s × )= µ × and a transcritical bifurcation at s • = s e when (1 + q θ )/δ = µ(s • ). The presence of multiplicative noise (i.e., with q θ = 0) induces a shift in the mode of the stationary PDF with respect to the deterministic steady-state solutions (i.e., with q θ = 0). This modified bifurcation behavior should be accounted for in reactor design and control.

Optimal operation
Given that as only meaningful open-loop operation point the locally asymptotically stable steady-state in the left branch of the curve C in Fig. 1 can be considered, and that this steady state is a direct consequence of the flow rate according to (11), the maximum conversion is determined by the solution of the optimization problem max qq (s e − x i1 ), s.t. (11) holds true.
As shown in [Schaum et al. 2012], the assocated optimal steady state substrate concentration s * = µ −1 (q * (1 + q θ )/(k 0 V )) (14) is located to the left of the saddle-node bifurcation point s × (cp. Fig. 1) and can be structurally stable [Andronov and Pontryagin 1937] with respect to slight fow rate fluctuations or structurally unstable and subject to disappearance due to bifurcation in consequence to stochastic variations in the flow rate, that manifest themselves in (14) by the shift produced by q θ > 0.

Fokker-Planck equation
The representation (9) underlines the structure of a diffusion-convection-reaction equation where in the multiplicative case the diffusion shows a spatial dependency and is coupled, while in the case of uncorrelated additive background noise the diffusion matrix is diagonal with constant diffusion coefficient. In accordance, the shape of the PDF will be different in the multiplicative and additve noise cases. Further note that according to (9) the velocity of the mode (most probable state) increases due to the drift force exerted by the auxiliary functions ϕ 1 , ϕ 2 (9b), (9c), implying that the presence of multiplicative noise goes along a faster PDF mode evolution in comparison to a non-multiplicative one (i.e., with q θ = 0), given that for q θ > 0 it always holds that ϕ i > f i , i = 1, 2.

Baysian formulation
In order to further highlight the transient and stationary behavior of the PDF π, following the approach in , consider the solution of the FPE in Bayesian form [Papoulis and Pillai 2002] as π(t, s, m) = κ(t, m|s)β(t, s), π(0, x) = π 0e (x) (15a) for x ∈ X e and t ≥ 0, with the conditional (κ) and marginal (β) PDFs, given by where it holds that β(t, s) = 0 for all s ∈ [s − , s + ] and s + s − β(t, s)ds = 1 for all t ≥ 0. Note that according to the preceding analysis the only dependency of the conditional PDF κ on the substrate concentration s is induced by the diffusion coefficients q 1j , j = 1, 2 in (9). Hence, for pure additive noise κ will be independent of s and its solution correspond to a Gaussian-like PDF with mode at s e .
In virtue of the boundary conditions, the dynamics of the marginal PDF β is given by With (15a) and substitution of (9d) one has Summarizing, the marginal PDF β satisfies the unidimensional FPE wherē For the stationary solutionβ it follows that withφ 1,∞ = lim t→∞φ1 (t, s), and C β so that The stationary solutionβ (18) shows substantial difference between the multiplicative and additive cases, coded in the relation between f 1 andφ 1 which depends on f 1 , q 11 ,κ and ∂ m q 12 . Particular differences will be highlighted below in the numerical simulation section.

Stochastic control
The consideration of the stochastic control problem is a relevant application-oriented subject [Åström 1970, Jazwinski 1970, Krstic and Deng 1998, Liu and Krstic 2012] that goes beyond the scope of the present study, and here it suffices to say that: (i) for a one-state isothermal reactor with Langmuir-Hinshelwood kinetics and OL multimodality or fragile monomodality, it has been established that proportional control can attain CL robust PDF monomodal behavior with mode close to the prescribed setpoint, and (ii) in this section, preliminary results on the attainment via proportional linear control of CL PDF behavior of the stochastic bioreactor (7d) (with two-state OL fragily stable monomdal PDF) are presented.
For this aim, let us consider the stochastic bioreactor (7d) with inlet flowrate linear proportional control u = −k(y −s), y = s.
(19) Given that the marginal PDF β for the substrate and the relation (19) between u and s, the associated control PDF is given by Driven by the substrate measurement interpreted as the most probable substrate value over the admissible probabilistic interval, the control must operate with (∼ r ) admissibly bounded offset and wide CL robust monomodal stationary PDF π(s, m) max β(t, s) ∼ rπ (s e , s * ), s * =s (21) with control setpoints at optimal deterministic value condition s * . In particular we are interested in: (i) the open-to-closed loop PDF spatiotemporal topology change, and (ii) the kind of statistical control-like [MacGregor and Kourti 1995] tradeoff between state PDF regulation, stochastic disturbance attenuation, and control PDF effort (in the sense of what the control PDF must statistically do to attain robust CL monomodality with admissible mode offset). The corresponding states (π) and control (ν) PDF evolutions are described by the closed-loop FPE (9) with u given by (19) that establishes [Papoulis and Pillai 2002] the control PDF ν(u, t) from π(t, s, m) according to (20) and the stochastic control equation.
From a previous control study [ Baratti et al. 2018] for a single-state reactor it is known that the solution of the CL FPE asymptotically reaches, with responses along deterministic and diffusion time scales (no metastability at play), a unique robustly monomodal stationary PDFπ with substrate mode (s m ) about the prescribed (deterministic OL fragile SS)s = s * if and only if the control gain k is chosen so that the closed-loop auxiliary equation (9b) with (19) evaluated for m = s e is non-decreasing in s.

NUMERICAL ILLUSTRATION
In this section the theoretical results obtained in the previous section are illustrated and corrobarated based on the numerical solution of the FPE based on a finite-volume method implemented as discussed in [Balzano et al. 2010].
The subsequent simulations emphasize the main changes to be expected in the passage from additive to multiplicative noise, namely a faster transient response and a reshaping of the stationary PDF. For the simulations the following set of kinetic parameters have been considered k s = 1, k i = 100.
(22) The optimal operation point corresponds toq/V = 1 and is close to the saddle-node bifurcation. In consequence it is not robustly stable and slight changes in the dilution rate can cause its dissappearance in a deterministic setup [Schaum et al. 2012], phenomenon referred to as "washout" [Bailey and Ollis 1986].

Open-loop behavior
Consider that the reactor is operating close to the deterministically locally asymptotically stable optimal operation point with an initial condition x 0 = [0.0901, 0.9] and subjected to random white noise fluctuations in the feed flow with intensity q θ = 0.004. The simulation results for the marginal PDF β defined in (15b) for this case are provided in Fig. 2, showing snapshots over time of the solution with q θ = 0 (mutiplicative noise -black lines) and q = 0.002 (additive noise -blue dashed lines). It can be seen in Fig. 2 that, (i) as predicted by the preceding theory, the mode time evolution with multiplicative noise is faster than the one with additive noise, (ii) in both cases the most probable state (largest mode of the PDF) converges towards washout (i.e., zero conversion), and (iii) during the transient the non-washout probability is higher for additive noise than for multiplicative noise. Note that fact (ii) illustrates the lack of robustness of the optimal operation point in presence of stochastic excitation, underlain by the deterministic structural instability with respect to small changes in the flow rate [Schaum et al. 2012].
Note further that the different transient shapes lead to different mode and mean values over time, implying important aspects that should be taken into account in a control application scenario (see also the discussion on mode versus mean control PDF control in  in the context of a single-state reactor).

Closed-loop behavior
In Fig. 3, the CL (top panel) and OL (bottom panel) marginal substrate state PDFs are reported for the deterministic optimal operation with initial conditions (m 0 , s 0 ) = (0.9, 0.0901), background noise intensity (q m , q s ) = (0.002, 0.002), inlet flowrate noise intensity q θ = 0.004, control gain k = 8 and set-points = 0.0901. As it can be clearly seen in Fig. 3, the proportional feedback control compensates the lack of robustness so that the most probable CL reactor state remains in close to (up to admissable offset) deterministic optimal operation in spite of the feed flow stochastic perturbation, while the most probable state of the OL reactor undergoes extinction. In Fig. 4, the steady state marginal substrate (top panel) and control variable (bottom panel) PDFs for the same condition as in Figure 3 but with two different control gains: k = 2 (blue line), k = 8 (black line) and OL (red dashed line) are reported. The steady state PDFs are monomodal but, for small k show a long right tail that become almost Gaussian increasing the control gain value as could be understood by comparing the mode (0.0921, 0.0908) and mean (0.1556, 0.0928) values for the two cases. The control effort (bottom panel) exhibits an opposite behavior i.e., increasing the control gain the distribution will widen with decreasing mode values. This is not surprising in the light of FP theory since, being the washout the most probable state for the selected operating conditions, the control effort is large and becomes larger as we require narrow substrate state PDF.

CONCLUSIONS
The effect of multiplicative stochastic excitation in the flow rate of a continuously stirred tank bioreactor with nonmonotonic Haldance growth rate is considered. Departing from the deterministic multiplicity and bifurcation behavior and the associated (potential) structural instability of the optimal, maximum conversion steady-state the effect of multiplicative versus additive noise is discussed in the framework of the Fokker-Planck equation. The impact of multiplicative noise under proportional feedback control is analyzed in terms of stabilization potential for the associated PDF versus energy wastefullness. The results extend the ones in ] to a two-state reactor.
The present study is a point of departure to address in the future the global-nonlinear state PDF estimation and control problems for stochastic chemical and biological reactors, with complex nonlinear deterministic dynamics.