{'fraction_non_alphanumeric': 0.042334062727935814, 'fraction_numerical': 0.01336250911743253, 'mean_word_length': 4.614414414414415, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Chance-Constrained Active Inference

Active Inference (ActInf) is an emerging theory that explains perception and action in biological agents, in terms of minimizing a free energy bound on Bayesian surprise. Goal-directed behavior is elicited by introducing prior beliefs on the underlying generative model. In contrast to prior beliefs, which constrain all realizations of a random variable, we propose an alternative approach through chance constraints, which allow for a (typically small) probability of constraint violation, and demonstrate how such constraints can be used as intrinsic drivers for goal-directed behavior in ActInf. We illustrate how chance-constrained ActInf weights all imposed (prior) constraints on the generative model, allowing e.g., for a trade-off between robust control and empirical chance constraint violation. Secondly, we interpret the proposed solution within a message passing framework. Interestingly, the message passing interpretation is not only relevant to the context of ActInf, but also provides a general purpose approach that can account for chance constraints on graphical models. The chance constraint message updates can then be readily combined with other pre-derived message update rules, without the need for custom derivations. The proposed chance-constrained message passing framework thus accelerates the search for workable models in general, and can be used to complement message-passing formulations on generative neural models.


Introduction
Similar to biological agents, learning to make decisions based on observations and feedback from the environment is also an essential task for autonomous artificial agents. Traditionally, adaptive linear control and model predictive control have been successfully applied in this area (Borrelli et al., 2017). Over the past few years, reinforcement learning has become the predominant approach (Recht, 2019). An emerging alternative perspective to decision making under uncertainty is active inference (ActInf) (Friston, 2010). ActInf is a neurosciencebased theory that has been used extensively to explain behavior of biological agents in dynamic environments (Friston, 2010).
ActInf is based in the free energy principle (FEP), and postulates that perception and action in biological agents minimize a free energy bound on Bayesian surprise. The free energy is an information-theoretic measure that bounds the current and the future expected statistical surprise, i.e., how unpredictable are the observations under a given generative model (GM). The free energy is associated with the Kullback-Leibler (KL) divergence (i.e., the distance) between the approximate and the true posterior. In particular, according to the free energy principle, the agent acts in such a way as to minimize a free-energy bound on the surprise, i.e., Bayesian surprise which, informally speaking, provides a quantification of the difference between the agent's predictions about the system behavior and the observed system behavior. Minimization of free energy is closely related to variational Bayesian methods, reinforcement learning (Sallans and Hinton, 2001;Tschantz et al., 2020;Sajid et al., 2021), and deep generative models (Ueltzhöffer, 2018;Fountas et al., 2020), another set of popular machine learning approaches (Goodfellow et al., 2014). ActInf is closely related to message passing on graphical models Friston et al., 2017), and several widely used message passing algorithms, including (loopy) belief propagation, variational message passing and expectation propagation can be derived as fixed-point equations of the (Bethe) free energy (Heskes, 2003;Yedidia et al., 2005;Dauwels, 2007;Zhang et al., 2017). This relation has been harnessed to develop elegant automated methods for ActInf (Schwöbel et al., 2018;van de Laar and de Vries, 2019).
In addition to investigation of motivating connections with the behavior of the biological systems (Friston et al., 2006;Ramstead et al., 2018), ActInf has been successfully utilized in applications in the traditional stochastic control scenarios, such as linear quadratic Gaussian (LQG) control and similar standard problems such as maze problems (Hoffmann and Rostalski, 2017;Ueltzhöffer, 2018;Schwöbel et al., 2018;Baltieri and Buckley, 2019;Millidge et al., 2020;Imohiosen et al., 2020), and exploration-exploitation balancing in multi-armed bandit problems (Markovic et al., 2021).
Despite these promising developments, the ActInf framework lacks certain desirable features present in model predictive control. In particular, there is no off-the-shelf standard ActInf formulation that allows inclusion of chance constraints in the problem setting. Chance constraints provide an attractive approach for on-line decision making for uncertain systems (Mesbah, 2016), i.e., systems where the dynamics are not fully known or the system contains certain components that are best modeled in a stochastic manner. In such settings, constraints on the system behavior, such as the agent remaining in a given region of the environment with a given probability, cannot directly be encoded in terms of prior beliefs. In contrast to approaches that constrain all realizations of the random variables, chance constraints allow for a (typically small) probability of constraint violation, which can significantly improve performance since chance constraints enable the decision maker trade performance with probability of constraint violation (Blackmore et al., 2011).
This paper proposes a computationally tractable approach to chance-constrained decision making, and applies it to an ActInf context. We include chance constraints in the ActInf objective (i.e., the free energy) by using the Lagrangian formalism. We then solve the Lagrangian optimization problem by variational calculus. Finally, we show that the proposed solution not only leads to a modular and scalable message passing framework for ActInf problems, but also provides a general purpose message passing framework that can account for chance constraints on graphical models in general. We claim the following main contributions: 1. We show that the analytic solution to the chance-constrained free energy problem yields posterior beliefs in the form of truncated mixtures. (Theorem 1) 2. We show how this solution can be interpreted in terms of message passing on a factor-graph representation of the generative model. (Theorem 2) 3. Consequently, our results provide a message passing framework that is specifically designed to account for chance constraints.
Message passing is inherently modular, and (variational) message update rules can be pre-derived and stored in a lookup table for later use (Korl, 2005;van de Laar, 2019). The chance-constrained message updates can then be readily combined with these pre-derived rules, without the need for laborious derivations. Our results illustrate that the proposed framework can successfully find solutions so that the rate of constraint violation specified in the original problem and the one that is actually observed during the closed-loop operation are close. The results also illustrate how to balance the constraints on the actions and the states through the usage of a tuning parameter, which enables exploration of different trade-offs between immediate and delayed intervention. otherwise. The model then factorizes as with non-negative real functions f a , and where x a ⊂ x collects the arguments of f a . In a probabilistic generative model, the individual factors usually represent conditional probability distributions. Probabilistic inference is then concerned with obtaining an (approximate) posterior belief q j (x j ) ∝ f (x)dx \j over a variable of interest x j , where x \j indicates the integration over all model variables except x j . We now briefly recap how the computation of these beliefs can be performed efficiently and automated over a factor graph (Loeliger et al., 2004), and how this process can be interpreted as a Bethe free energy minimization problem (Yedidia et al., 2005). With these concepts firmly in place, we move to chance constraints and the formal problem statement in Sec. 2.4.

Factor Graphs for Marginal Belief Computation
A factor graph can be used to visually represent a factorized function. In this paper we use the bi-partite factor graph representation. A bi-partite factor graph consists of variable-nodes V, factor-nodes F, and edges E that connect variablenodes with factor-nodes. A variable-node i ∈ V is connected to a factor-node a ∈ F by an edge (i, a) ∈ E if (and only if) the variable x i is an argument of the factor-function f a . An example section of a graph is drawn in Fig.1, where the circle and square represent a variable-and factor-node respectively. We write Figure 1: Bi-partite subgraph of a model around a variable-node j (circle) and factor-node b (square), with indicated messages. Ellipses represent a continuation of the model. the neighborhood of a variable-node i as F(i), which collects all factor-nodes in F that are direct neighbors of i. Similarly, V(a) collects all variable-nodes in V that are direct neighbors of a. Suppose we are interested in obtaining a posterior belief q j (x j ). The belief propagation algorithm (Pearl, 1982) then prescribes we send messages from the branches of the graph towards the variable-node of interest, following the recursive application of the belief propagation update rules: where x b\j collects all x b with the exception of x j . Here, µ jb (x j ) represents the message from a variable-node j ∈ V to a neighboring factor-node b ∈ F(j); and reversely for µ bj (x j ). These messages are illustrated in Fig. 1. The posterior belief can then be expressed as with Z j = µ jb (x j )µ bj (x j ) dx j a normalizing constant. In practice, for numerical stability, messages are often re-normalized after computation. Furthermore, messages are usually scheduled for computation, and are often referred to by their position in the schedule instead of their location in the graph. We will use a similar notation in Sec. 4. See (Bishop, 2006) for a more detailed introduction to (approximate) inference on bi-partite graphs.

Bethe Free Energy Interpretation
The Bethe free energy for a factorized model of the form of (1) is defined as where d i represents the degree of variable x i . Here U a [q a ] = − q a (x a ) log f a (x a ) dx a denotes the average energy for factor f a , and H[q a ] = − q a (x a ) log q a (x a ) dx a denotes the entropy. The Bethe free energy is optimized with imposed normalization and marginalization constraints: such that the q a and q i represent (approximate) posterior probability distributions (beliefs).

Free Energy Minimization for Active Inference
Active Inference usually defines dynamic models that specialize variables into parameters, states, observation and control sequences for past and future times. Free energy minimization for ActInf is then presented as a dual objective, where minimization of free energy for a model of past variables accounts for state and parameter estimation (perception), and free energy minimization of free energy for a model of future variables accounts for policy planning (Baltieri and Buckley, 2018;. In the present paper we assume that the current state is observed and that model parameters are given. Therefore, this paper only concerns inference for policy planning. Extensions for perception are however straightforward. Chance constraints only affect inference for planning, and therefore standard techniques for state estimation and parameter learning can be employed (van de Laar and de Vries, 2019).
Furthermore, the current paper employs the Bethe Free Energy (BFE) formulation (4) for policy planning (Schwöbel et al., 2018;van de Laar and de Vries, 2019) instead of the more traditional Expected Free Energy (EFE) (Friston et al., 2015). The BFE is known to lack the epistemic qualities of the EFE (Schwöbel et al., 2018), which can be compensated for by introducing an additional mutual information term between the states and the observations to the BFE objective (Parr and Friston, 2019). The benefit of the uncompensated BFE however, is that traditional message passing algorithms, including (loopy) belief propagation, variational message passing, expectation propagation and generalized belief propagation algorithms can all be derived as fixed-point equations of the variational free energy by the use of variational calculus, see (Yedidia et al., 2000;Heskes, 2003;Yedidia et al., 2005;Dauwels, 2007;Zhang et al., 2017).

Chance Constraints
A chance constraint imposes that the probability mass of a belief q j (x j ), j ∈ V outside of a 'safe' region S j ⊂ X j cannot exceed a pre-set threshold ∈ [0, 1]. Formally, a chance constraint imposes the inequality with Our problem statement then becomes two-fold, namely: 1. Find the stationary points of the Bethe free energy (4) under the normalization and marginalization constraints of (5) and chance constraints of the form (6) (Theorem 1); 2. Interpret the retrieval of stationary points of the chance-constrained Bethe free energy as message passing on a factor graph (Theorem 2).
The simulations of Sec. 4 further specialize the model variables into state, observation and control sequences and demonstrate the added value of chance constraints in an ActInf setting. Crucially, with an interpretation of chance constraints in terms of message passing on a factor graph, chance constraints can be readily applied to any factorized model. Formulating chance constraints as a click-on module for approximate inference then greatly improves the application range of chance constraints.

Chance-Constrained Message Passing
In this section we formulate the method of chance-constrained message passing. We identify the stationary points of the chance-constrained Bethe free energy and interpret the result in terms of message passing on a factor graph. We work towards a practical message-passing update rule for chance-constrained variables, as summarized in Algorithm 1. A brief introduction to variational calculus is available in Appendix A. Proofs can be found in Appendix B.

Stationary Points
From the Bethe free energy (4) and the constraints of (5), (6), we can construct the Lagrangian where the Lagrange multipliers γ, ζ, η enforce the constraints of (5), (6). Under strong duality, for the inequality constraint in (6) we have the complementary slackness condition (Boyd and Vandenberghe, 2004, Ch. 5). This condition states that for optimality we have Therefore, either η i > 0, which implies that the chance constraint of (6) holds with equality (active) or η i = 0, which implies that the chance constraint may hold without equality (inactive). In other words, the complementary slackness condition requires us to consider two scenarios: i) (6) holds with equality for η i > 0 and ii) (6) is satisfied under η i = 0. Hence, if η i > 0, the chance constraint is activated and enforced with equality.
In Lemmas 1, 2 we express the stationary points of L[q] in terms of the beliefs. The proofs are presented in Appendix B.1 and Appendix B.2. Lemma 1. Stationary points of (7) as a functional of q b , b ∈ F, are of the form with Proof. See Appendix B.1.
Note that the µ ib have not yet been identified or interpreted as messages. We will explicitly make this connection in Sec. 3.3.
Lemma 2. Stationary points of (7) as a functional of q j , j ∈ V, are of the form with a normalizer that still depends on η j .
Note that, in contrast to (3), this result incorporates an additional exponential term for η j . We will identify this multiplier in Sec. 3.2. However, we already know that when the chance constraint for j is inactive, hence η j = 0 as a consequence of the complementary slackness condition. In this case, (9) reduces to (3).

Active Chance Constraint
In this section, we identify the stationary points under active chance constraint. The result is stated in Theorem 1.
Theorem 1. Under active chance constraint, stationary points of (7) as a functional of q j , j ∈ V are of the form with This remarkable result tells us that the corrected belief q * j (x j ; η j = η * j ) is obtained by scaling the probability mass of the uncorrected belief q j (x j ) over the respective safe and unsafe regions. This defines the corrected belief as a mixture of truncated beliefs. The optimal scaling of (10) ensures that the overflow is equal to .
The complementary slackness condition ensures that the chance constraint is only enforced if the probability mass of the unconstrained belief overflows the 'safe' region S j by more than ; i.e., the uncorrected belief is 'unsafe' when where we refer to Φ (0) j as the 'safe mass'.
If (12) is satisfied, then the posterior density q j (x j ) is corrected according to (10), which 'pushes' the probability mass (just) back inside the safe region.

Chance-Constrained Message Passing
In this section, we show that chance constraints (10) can be interpreted as auxiliary factor-nodes (with a specific node-function), and can be enforced by belief propagation in an augmented graph.
Theorem 2. Given a bipartite graph G = (F, V, E) with a variable node j ∈ V, and an associated Bethe free energy (4) with a chance constraint (6) on the belief q j (x j ). Then, stationary points of (7) can be obtained by belief propagation on an augmented graph G = (F , V, E ), where and auxiliary node function Proof. See Appendix B.4.
Theorem 2 shows that chance-constrained message passing can be seamlessly incorporated within the belief propagation framework. Chance constraints simply enter the model definition as auxiliary factors, whose factor function depends upon the incoming message, see Fig. 2. Because uncorrected belief (11a) is being represented by the (re-normalized) incoming message µ jg (x j ), this allows for a modular application of chance constraints by augmenting the original graphical model with auxiliary nodes.

Gaussian Approximation
Since the message µ gj (x j ) introduces discontinuities, the computations for dependent messages may grow prohibitively complex. For efficient computations, it can be helpful to make a Gaussian approximationq j (x j ) to the corrected belief q * j (x j ; η j = η * j ), e.g., by moment matching. The resulting (approximate) message then follows from If the message µ jg (x j ) is also Gaussian, this computation is easily performed by subtracting the canonical statistics. This procedure then resembles the expectation propagation algorithm (Minka, 2001;Cox and de Vries, 2018). Interestingly, the expectation propagation algorithm can also be derived in terms of Bethe free energy optimization, where the marginalization constraints (5a) are replaced by moment-matching constraints (Zhang et al., 2017). This makes the Gaussian approximation consistent with the Lagrangian approach as presented in this paper.
The approximated beliefq j (x j ) however renders the chance constraint (6) inexact. As a result, the approximated belief needs to be iteratively re-corrected: where n denotes an iteration counter. This leads to the procedure summarized in Alg. 1, and depicted in Fig. 3.
Algorithm 1 Chance-constrained message passing with Gaussian approximation Given a Gaussian inbound message µ jg (x j ) Compute the uncorrected belief q With this algorithm, we have derived a practical chance-constrained message update from the first principles. The message update can be readily applied to any continuous variable that requires a chance constraint. Note however, that when multiple chance constraints are imposed on the model, the message passing algorithm itself becomes an iterative procedure because of circular message dependencies. For example, a message incoming to an auxiliary node g might (indirectly) depend on a message that exits another auxiliary node h. In turn, this exiting message depends on the incoming message to h (1), which depends on the message exiting g, etcetera. In order to break this circular message dependency, uninformative messages can be used to initialize the algorithm.

Simulations
In this section we simulate a drone that aims to elevate itself above a given height threshold with a preset probability, under the influence of a stochastic vertical wind. We define the drone elevation level over time by x = {x 0 , . . . , x t , . . . , x L }, x t ∈ R, and actions (ascension velocity) a = {a 0 , . . . , a t , . . . , a L }, a t ∈ R. A timedependent m w,t defines the expected wind velocity that acts upon the agent. The discrete-time stochastic system is defined as: where v w defines the wind velocity variance. We define an agent that directly observes its elevation level and has knowledge of the statistical system properties m w,t and v w . The agent models future states of the system with a fixed time horizon T . As a shorthand notation, we write the future (including current) states x t = {x t , . . . , x t+T } and control variables u t = {u t , . . . , u t+T −1 }. For notational convenience, we drop the t subscript from these collections. The agent model at time t is defined as: with a respective state transition model and control prior We factorize and constrain the variational posterior distribution such that (van de Laar and de Vries, 2019) where x \t indicates the collection of latent states (the state sequence x without the observed current state x t ). The goal of the agent controller then becomes to find the policy π t = {a t , . . . , a t+T −1 } that minimizes the Bethe free energy under the normalization and marginalization constraints of (5) and chance constraints where the safe region S = (1, ∞) and violation probability are identical for all future state variables.

Graphical Model and Schedule
As detailed in Sec. 3, Bethe free energy minimization under chance constraints can be performed by message passing on an augmented model. The graphical representation of the augmented model is depicted in Fig. 4.
. . . . . .  (16), with p x,k expanded according to (17) (dashed rectangle), and indicated forward (numbers) and backward (letters) message passing schedules for optimization of (19). Circle and square nodes indicate variable-and factor-nodes respectively. Dark nodes indicate observed variables or fixed parameters, and auxiliary factor-nodes (14) are dashed. Ellipses indicate a continuation of the model. Dark messages are computed by the variational update rule, see (Winn and Bishop, 2005;Dauwels, 2007).
The schedule comprises a forward-backward scheme, as illustrated in Fig. 5. Four message updates in Fig. 5 are of particular interest. Firstly, since (18) constrains the belief over controls to a point-mass, it follows that where i counts the number of schedule (forward-backward) iterations. The schedule is initialized with a (0) B (x k+1 ) takes on the role of µ jg (x j ) in Alg. 1. Because the noise in the model is Gaussian, this message will be an (unnormalized) Gaussian as well. Therefore, by application of Alg. 1, the third message of interest, µ (i) 6 (x k+1 ) is computed. For the initial forward pass, µ B (x k+1 ) = 1 is considered uninformative. Fourthly, µ (i) F (u k ) carries information upward to the control variables. Because the variational posterior is chosen to factorize between the state and control sequence (18), the µ (i) F (u k ) message is computed by a variational update rule as detailed in (Winn and Bishop, 2005) and (Dauwels, 2007).
The action for the next iteration then follows from Iterating the schedule then corresponds with an expectation maximization scheme. The expectation step of this scheme computes the µ

Control Law
Note that the Bethe free energy of (19) is still a function of the observed current elevation x t . We can then evaluate the optimal action a t as a function of the current elevation x t (the control law), for a given wind profile, chance constraint and model parameters. In order to gain an intuition for controller behavior, we fix m w,t = 0 for all t. We plot the control law in Fig. 6, for varying values of the lookahead horizon T , chance constraint threshold , wind variance v w and control prior precision λ.
The top-left diagram shows that with growing lookahead horizon T , the agent starts intervening at higher elevation. With this anticipatory effect the agent prepares for events in the more distant future. The top-right diagram also shows that the agent intervenes at higher elevation with decreasing . When violation of the constraint grows less desirable, the agent must intervene earlier in order to assure that sufficient probability mass is present in the safe region. Also note that no further action is proposed beyond an intervention threshold. Once the agent is sufficiently elevated, no corrections are proposed until the agent wanders (or is forced) below the intervention threshold. The bottom-left figure shows a similar effect for growing wind velocity variance v w . When the system grows more stochastic, chance constraint abidance is ensured by intervening at higher elevations. Finally, the bottom-right figure illustrates what happens when the chance constraint is combined with a Gaussian prior constraint on control. Increasing the control prior precision λ penalizes immediate correction. For low precisions (low penalty on control magnitude), the slope of the control law below the intervention threshold is equal to 1, and compensation is immediate. Control grows more robust with growing precision, at the cost of prolonged chance constraint violation.

Comparison Against a Goal-Driven Agent
In order to illustrate the difference in behavior between a chance-and a goaldriven ActInf agent, we compare the results of Fig. 6 with an ActInf agent where the chance constraint is replaced by a goal prior. We use the graphical model definition of Fig. 4 and define the auxiliary node function as a fixed prior f x,k+1 (x k+1 ) = N (x k+1 |m x , ϑ x ) for all t ≤ k ≤ t + T − 1. We choose m x = 2, and the variance ϑ x = 0.18478 such that the overflow of the safe region 1 − S f x,k+1 (x k+1 ) dx k+1 ≈ 0.01 resembles the situation for = 0.01. The message passing schedule then follows the definition of Fig. 5, where µ (i) 6 (x k+1 ) is no longer computed by Alg. 1 and propagates the fixed goal prior instead. Fig. 7 shows the resulting control law for m w,t = 0, T = 1, v w = 0.2 and varying λ.
The results of Fig. 7 show that the control for the goal-driven agent grows more robust with increasing λ -similar to the control law for the chance-driven agent (Fig. 6, bottom right). For the smallest λ, the control law for the priordriven agent resembles the corresponding control law for the chance-driven agent (dotted curve) only for elevations x < 2. For elevations x > 2, the goal-driven agent proposes downward corrections, while the chance-driven agent proposes no corrections. This comparison illustrates how a chance-driven agent avoids unnecessary interventions.

Simulation Results
In this section we study an active inference agent in interaction with a simulated environment. The action-perception loop is based on (van de Laar and de Vries, 2019) and consists of four steps at every time t: 1. Observe the current agent elevation; 2. Infer a policy from the current elevation and the future expected wind velocities by chance-constrained message passing; 3. Act by selecting the first (current) action from the inferred policy; 4. Execute the selected action in the system and advance the time index by one.
The results for ten thousand independent runs are plotted in Fig. 8 for a chance-driven agent (left) and a goal-driven agent (right). The first row of diagrams plots the expected wind velocity over time, which is identical for each run. The sampled wind velocity trajectories w t do vary per run, under influence of the wind velocity variance v w . For 5 ≤ t < 10 a downward draft attempts to push the drone below the minimal safe elevation (dashed). The second row plots the drone elevation trajectory for a randomly selected subset of runs. Corresponding actions are plotted in the third row. The fourth row evaluates the relative number of runs that violate the safe-zone over time.
It can be seen that both agents undertake corrective actions in order to compensate for the downward wind. However, while the chance-driven agent (left) only proposes upward corrections below the intervention threshold, the goal-driven agent (right) proposes additional downward corrections above the threshold. Furthermore, it can be seen that the maximal empirical violation for the chance-constrained agent mostly remains below the chance constraint target violation probability of = 0.01 (dashed), while the goal-driven agent systematically overshoots the target violation probability, i.e. violates the chance constraint. Compared to the chance-driven agent, the maximal empirical violations for the goal-driven agent are also larger. This effect can be explained in terms of the constrained beliefs. Namely, the chance-driven agent constrains the posterior beliefs, while the goal-driven agent imposes prior constraints on the model. Prior constraints may still be violated by the corresponding posterior beliefs, leading to more pronounced empirical violations.

Conclusions
In this paper, we formulated chance-constrained optimization of the Bethe free energy in terms of message passing on a factor graph. We showed that, in the factor graph representation of the generative model, chance constraints can be imposed by auxiliary factors that force (a specified portion of) the probability mass of the chance-constrained beliefs inside a designated safe-zone. Message passing on the augmented graph, with the auxiliary factor-nodes included in the graph, then automatically balances the imposed chance constraints with additional (prior) constraints on the generative model. Chance constraints can thus be interpreted as modular click-on extensions to the generative model, similar to conventional factor-nodes (Loeliger et al., 2004), and can thus be used to complement message-passing formulations on generative neural models van de Laar et al., 2018).
However, because the analytical result for the chance-constrained update includes an inherent discontinuity, direct application of this rule may still lead to message updates that grow prohibitively complex. To remedy this, we proposed an algorithm that approximates the resulting message with a Gaussian form. This algorithm offers a tractable formulation of chance-constrained message passing. The proposed message passing interpretation of chance constraints then vastly enhances the modularity and flexibility of chance-constrained inference, and can accelerate the search for workable models (Blei, 2014).
We demonstrated chance-constrained message passing in the context of active inference. We compared the simulated behavior of a chance-driven agent with a goal-driven agent, where the chance constraints are replaced by traditional prior beliefs on future outcomes. The results illustrate how the goal-driven agent continually proposes corrections, whereas the chance-driven agent seizes interventions above a threshold. Chance-constrained ActInf may thus avoid unnecessary interventions and reduce the cost of control.
The results for the chance-driven agent showed that, in the absence of additional prior constraints, the empirical chance constraint violation ratio mostly remains below the pre-set target violation probability. An added prior constraint on controls robustifies control at the cost of prolonged chance constraint violation. Chance-constrained active inference thus weights all imposed constraints on the generative model, allowing e.g., for a trade-off between robust control and empirical chance constraint violation.
functional L[q]. We define an infinitesimal variation of q by where β → 0, and φ(x) is a continuous and differentiable "test" function.
The functional derivative δL/δq relates a variation in q to a change in L, by (Parr, 1980): The procedure then becomes to apply the operations on the l.h.s. to L, and bring it into the form of the r.h.s., which allows us to identify the functional derivative δL/δq. The stationary points q * are then obtained by setting δL/δq ! = 0 and solving for q.

B.1 Proof of Lemma 1
Application of (20) to (7) as a functional of q b , yields Identifying the functional derivative δL[q b ]/δq b and setting it to zero, we obtain We now define µ(x j ) = exp ζ(x j ) and apply the normalization constraint, which recovers (8).

B.2 Proof of Lemma 2
Application of (20) to (7) as a functional of q j , yields Identifying the functional derivative δL[q j ]/δq j and setting it to zero, yields which is the first expression for q * j . We can obtain a second expression for q * j by applying the marginalization constraint to the result of Lemma 1. Substituting (8) in (5a), where we identified a new quantity µ bj (x j ) (note the reverse indexing). Interestingly, the marginalization result of (23) not only holds for the specific factor b, but for all factors that neighbor j. Therefore, by symmetry, we can iterate the relation of (23) for all c ∈ F(j): We choose to exclude b itself from the iteration on both sides, and obtain We substitute (22) in the l.h.s. of (24), and note that the product on the l.h.s. now has d j − 1 terms, and that neither of these terms depend on c. This allows us to remove the d j − 1 terms from the exponent of (22), which yields exp(1 − d j + γ j + η j g j (x j )) a∈F (j) µ ja (x j ) = c∈F (j) c =b 1 Z c µ jc (x j )µ cj (x j ) .

B.3 Proof of Theorem 1
We start from (9), and use the definitions of (11) to obtain We have now identified the η j multiplier. Substituting this result back in (9) recovers (10), which expresses the corrected belief in terms of the uncorrected belief.

B.4 Proof of Theorem 2
From (9), we express the uncorrected belief in terms of the messages with Z (0) j = Z j (η j = 0). We now construct the augmented graph G = (F , V, E ) according to (13), and define a message with f g (x j ) as defined by (14).
Substituting (26) and (27) in (10) then yields the corrected belief in terms of the messages with µ jg (x j ) = a∈F (j) a =g µ aj (x j ) .