{'fraction_non_alphanumeric': 0.04440432108210668, 'fraction_numerical': 0.008079077026654833, 'mean_word_length': 4.3790946502057615, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Accurate computation of conditional expectation for highly non-linear problems

This paper focuses on inverse problems to identify parameters by incorporating information from measurements. These generally ill-posed problems are formulated here in a probabilistic setting based on Bayes's theorem because it leads to a unique solution of the updated distribution of parameters. Many approaches build on Bayesian updating in terms of probability measures or their densities. However, the uncertainty propagation problems and their discretisation within the stochastic Galerkin or collocation method are naturally formulated for random vectors which calls for updating of random variables, i.e. a filter. Such filters typically build on some approximation to conditional expectation (CE). Specifically, the approximation of the CE with affine functions leads to the familiar Kalman filter which works best on linear or close to linear problems only. Our approach builds on a reformulation, which allows to localise the operator of the CE to the point of measured value. The resulting conditioned expectation (CdE) predicts correctly the quantities of interest, e.g. conditioned mean and covariance, even for general highly non-linear problems. The novel CdE allows straight-forward numerical integration; particularly, the approximated covariance matrix is always positive definite for integration rules with positive weights. The theoretical results are confirmed by numerical examples.


Introduction
Inverse problems. This paper is focused on the identification of parameters, given by a random variable Q : Ω → Q, using measurements. This problem has an enormous importance in engineering practice; unfortunately, the parameters usually cannot be observed directly but only indirectly as a response of some system, given by an observation operator Y Q : Q → Y. It is therefore called an inverse problem [18] contrary to the forward problem Y Q . Moreover, the measurements are also polluted by some error distributed in accordance to a random variable E : Ω → Y. The measured valueŷ ∈ Y is then the sample of observation operator where we assume an additive error; for concrete problem see Example 2. Inverse problems are usually ill-posed because an observation can lead to many parameters satisfying (1), the parameters q can be very sensitive to observationŷ, or even the observationŷ can be out of the range of the observation operator Y Q . This mathematically manifests itself in the fact that usually Y Q is not invertible. Bayesian updating. The probabilistic description of inverse problems is an elegant approach that relies on Bayes's theorem [3,13,17]. Modelling the prior parameters q = Q(ω) as uncertain, the ill-posed inverse problem turns out to be well-posed with a unique (stochastic) solution. Therefore the updated (or assimilated) parameters can describe all possible parameters (along with their probabilities) satisfying the inverse problem (1) in a stochastic sense. Many Bayesian approaches to inverse problems employ the updating in terms of probability measures or corresponding densities. However, the response of the system (forward problem), from which the measurements are taken, is typically expressed as a random variable (RV), which makes repeated updating with probability densities a non-straightforward computation.
Conditional expectation. The Bayesian updating in terms of random variables builds on the concept of conditional expectation (CE), which is defined as another RV. However, its approximation is a difficult task. The approximation of a CE with affine (linear) mappings leads to the familiar Kalman filter (KF) [8] and many of its sampling variants such as the Ensemble KF [5,4], Extended KF [7], or accelerated KF employing a surrogate model for the forward problem [9]. There is also a sampling-free variant of the filter called polynomial chaos Kalman filter [1,12,14,15] when all random variables are expressed in terms of polynomial chaos expansion (PCE) and only their coefficients are updated. However, all the variants provide reliable results that match the conditional distribution only for linear or close to linear forward problems. The distribution of the filtered RV is duly exact for Gaussian linear problems, and deviates from it more and more as this condition is not met. A better outcome can be achieved with a discretisation of the CE by higher order polynomials [11,10], however its approximation properties are still poor for general non-linear problems, see Section 2.5 for details.
Novelty. The conditional expectation Φ Q|Z is a map (from measurement to parameter space), which is generally very difficult to approximate. For calculating the posterior quantities one needs only a value of the CE at the point of observationŷ. Here we focused on the novel evaluation of this value Φ Q|Z (ŷ) after the observationŷ, called here conditioned expectation (CdE), using the variational formulation of CE. It overcomes the limitations of previous approaches and leads to an accurate and more robust method presented in Section 3.
Organisation of the paper. The target of this paper is to provide, besides new results, a variational view on CE from the computational perspective. Because of various notations and approaches to CE, this paper is designed to be self-contained, including many proofs or their outlines that can help in understanding. Particularly the existing theory about CE, conditional probabilities, and conditional probability densities is described in Appendix A. Then the main text contains a brief version in Section 2 which is accompanied by a model inverse problem in a Bayesian setting and also by existing filters, e.g. the Kalman filter. The novel development and numerical approximation of conditioned expectation is covered in Section 3. The results are confirmed by numerical examples in Section 4, and their summary can be found in the conclusion in Section 5.

Inverse problems via Bayesian updating
This section is focused on inverse problems using Bayesian updating. The basic notation and definitions of random variables and probability distributions in Section 2.1 is followed by the description of a model problem in Section 2.2. Bayesian updating for probability density functions (PDF) and conditional expectation is summarised in Sections 2.4 and 2.3; the full version of this topic is covered in Appendix A. The last Section 2.5 of this part is devoted to Bayesian filters along with critical discussion about its approximation.

Random variables and probability distributions
The definitions and notations about random variables and probability distributions are introduced here. Let (Ω, S, P) be a probability space with sample space Ω, set of events S as a σ-algebra, and probability measure P : S → [0, 1]. The parameter space Q (and also the measurement space Y) will be assumed to be a finite dimensional vector space with a scalar product u, v Q (or u · v) and induced norm u Q = √ u · u. It is considered to be a measurable space (Q, B Q ) with Borel σ-algebra B Q (or simply B) generated by the open sets in Q.
A measurable function X : Ω → Q is called a random variable (RV). The space L p (Ω, S, P; Q) (or simply L p (Ω; Q) or L p ) is the collection of (equivalence classes of) RVs with finite norm is defined as a duality pairing f, g L p ×L q = Ω f (ω)g(ω)P(ω), and it becomes a scalar product f, g L 2 ×L 2 = f, g L 2 for p = 2, and L 2 is a Hilbert space. The expectation E : L p (Ω, S, P; Q) → Q is denoted as E X = X = Ω X(ω)P( dω). The characteristic function 1 A of a set A ⊂ Ω is a function that equals 1 on A and zero otherwise. The composition of two random variables

The model problem
The model problem focuses on the identification of parameters q ∈ Q, which are initially described with a distribution determined by a random vector Q ∈ L 1 (Ξ, S Ξ , P Ξ ; Y) with q = Q(ξ), or by a probability density function (PDF) f Q . Unfortunately, the parameters q can usually not be observed directly but only a response of it given by the observation operator Y Q : Q → Y.
Here the observation space is typically a finite dimensional vector space R m , which corresponds to the information obtained from m measurements. The measurement operator thus generates a measurement random vector Y ∈ L 1 (Ξ, S Ξ , P Ξ ; Y) defined as However, the measurement is typically influenced by a measurement error e = E(θ) ∈ Y with a distribution given by a random vector E ∈ L 1 (Θ, S Θ , P Θ ; Y) or probability density f E , which has usually zero-mean and a Gaussian distribution. The measurement (2) accompanied by the error establishes a random variable (measurement with error) form the space L 1 (Ω, S, P; Y) with the product space Ω = Ξ × Θ, product σ-algebra S = S Ξ × S Θ , and product measure P = P Ξ × P Θ , which means that the error E is independent of the parameter Q. For reasons of simplicity we talk about Z as the measurement.
The random variable Q, defining the distribution of the parameters, is primarily defined on Ξ. We will inject that to Ω = Ξ ×Θ by definingQ(ξ, θ) = Q(ξ); for notational simplicity we will use only one symbol Q, so Q ∈ L 1 (Ω, S, P; Q) as well as Q ∈ L 1 (Ξ, S Ξ , P Ξ ; Q). The structure of the measure spaces and maps is summarised in the following diagram.
The conditional expectations Φ Q|Z : Y → Q of the parameter Q and Φ X|Z : Y → X of a general RV X w.r.t. the measurement Z are discussed in Section 2.3 and Appendix A.

Problem 1.
The goal is to calculate the quantities of interest (mean, covariance, etc.) of parameters q ∈ Q with the information from the measurement, given by the observed valuê y ∈ Y and observation error E. Particularly the vectorŷ, usually from Y = R m corresponding to m measurements, is the value taken at the measurement site with the error given by E.
Example 2 (Identification of loads). As an example, one can consider identification of magnitude of heat source by measuring the response (temperature u) of the stationary heat transfer described by with some boundary conditions (possibly nonlinear), where f i are loads and q i are uncertain quantities for i = 1, . . . , n. The information about the system is given by measuring the temperature at measurements points x i for i ∈ {1, . . . , m}, which corresponds to an observation operator Y Q (q) = [u(x 1 ), u(x 2 ), . . . , u(x m )] for u obtained from model (5).

The conditional expectation
In a probabilistic setting, Problem 1 to estimate parameters from the measurement is based on Bayes's theorem. Mathematically, it is formulated for two events A and B as subsets of Ω where P(A) is the probability of event A, P(A ∩ B) is a probability of simultaneous occurrence of events A and B, and P(A|B) is a conditional probability, i.e. a likelihood of event A given that event B has occurred. However, this is well-defined only when the probability of event B is positive (P(B) > 0). One has to overcome this limitation as well as the fact that Bayes's theorem estimates the posterior probability only w.r.t. a single observed event. However one needs to incorporate the new information of the whole set of events. The proper approach is the conditional expectation w.r.t. to a random variable, which is covered in many monographs, e.g. [13,2], and also in Appendix A.
Here conditional expectation is summarised for a prior random variable Q ∈ L 1 (Ω, S, P; Q) with respect to a given measurement Z ∈ L 1 (Ω, S, P; Y). However, there is a natural question what one can observe from measurement Z and how it can be mathematically formulated. Assuming an event B in Y has been observed, then the corresponding probability of that is expressed with the help of the push-forward measure In other words, all events in parameter space Ω that lead to an event B lie in the preimage Z −1 (B); particularly, the preimages in the example visualised in Figure 5 are exactly the neighbourhoods of the plotted contours. Therefore, the information from the measurement Z that can be obtained about the parameters is characterised by all preimages in Ω for all possible sets B, i.e. the σ-algebra generated by the measurement Z denoted as σ(Y ), see (20) for a definition.
The conditional expectation of any random variable X : Ω → X , which could be a function of Q, w.r.t. the measurement Z ∈ L 1 (Ω, S, P; Y) is then defined as a random variable Φ X|Z : Y → X such that it satisfies the orthogonality condition The conditional expectation exists for all parameters X in L 1 (Ω, S, P; X ), which can be proven by the Radon-Nikodým theorem [13], and is also unique as an element in L 1 (Y, B, P Y ; X ) (i.e. it is unique up to the set of P Y -measure zero). When the parameters X have a finite variance, i.e. X ∈ L 2 (Ω, S, P; X ), the CE is a projection as it minimises the mean square error the variational equation of which is (6) in the L 2 × L 2 duality.

Bayesian updating using probability densities
Bayesian updating using probability densities is presented here, for details see Appendix A.2.
Particularly taking an open set A in the parameter space Q, the conditional probability distribution P Q [A|Z] is defined as the conditional expectation w.r.t. a characteristic function of the preimage of A, i.e.
As it is a probability measure on the space Q, it can (in some cases) be expressed with a conditional probability density function f Q|Z : When the conditional probabilities are expressed in terms of probability densities we obtain a variant of Bayesian updating where f Q (q) is the probability density of prior random variable Q, f Z|Q (ŷ|q) is a likelihood, and the evidence Z s = Q f Z|Q (ŷ|q)f Q (q) dq ensures that the whole conditional probability equals one, i.e. that Q f Q|Z (q|ŷ) dq = 1. For the observation with additive error (3), the likelihood can be expressed in a special form for details see Lemma 18. Altogether we receive the Bayesian updating in the final form From PDFs one can calculate the mean and variance of the posterior (conditional) distribution For a future comparison to the conditioned expectation to be developed, we stress here that the conditional mean and covariance depend on the observation valueŷ.

Filtering
The whole aim of updating (filtering) the parameters is to obtain a posterior RV with updated (or assimilated) distribution w.r.t the measurement Z and observed valueŷ ∈ Y. Here several existing filters are discussed. Certainly, the filter has to contain the conditional expectation Φ Q|Z (ŷ), which predicts the posterior mean for observationsŷ, and a zero-mean random variable R determining the posterior distribution. Naturally, the filter should be a function of the prior but also of the measurement; however, its effective determination remains an open problem. One approach is based on the orthogonal decomposition of the prior RV , which leads to a so-called CE mean filter Although this is exact for the specific setting of normally distributed prior and linear observation Z, it fails to predict correct distribution (except mean) for general problems for all possible observationsŷ. Particularly the covariance C QaQa = C RR = E R ⊗ R and also higher moments of posterior RV Q a are determined by the zero-mean part R, which in (11) is independent of the observation valueŷ, see also Section 2.5.2 with a numerical example. Nevertheless the correct covariance can be obtained, similarly as the mean, from the conditional expectation C = ΦQ ⊗Q|Z (ŷ) but with respect to the RVQ ⊗Q forQ = Q − Φ Q|Z (ŷ). It allows to scale the CE mean filter (11) to obtain a filter which can have correct mean and covariance where the inverse is meant in the sense of pseudo-inverse. Still the filter cannot have correct distribution for general problems. Actually any filter expressed as a mean plus zero-mean random variable R = f (Q, Z) depending on the prior and the measurement provides a filter with correct mean and covariance. The novel filters and their approximations will be studied in following publications.

Approximation of conditional expectation
The filters are difficult to realise numerically because they are based on an unknown measurable map of conditional expectation Φ Q|Z : Y → Q. A simple approximation is an affine approximation of CE as The optimal matrix K coincides with the Kalman gain, explicitly expressed as K = C QZ C −1 ZZ using the covariance matrices C QZ and C ZZ of RVs Q and Z; the inverse is considered to be a Moore-Penrose pseudo-inverse. This Kalman filter (13) has several sampling variants such as the Ensemble Kalman filter [4]. On the other hand, when the RVs Z, Q, and Q a,1 are expressed in terms of a polynomial chaos expansion (PCE) -i.e. as a polynomial approximation of a basic random variable -the PCE variant of the filter is obtained [1,12,14,15]. A more complex approach is based on approximation of CE with multivariate polynomials Φ Q|Z ≈ Φ k Q|Z of order k, obtained by the minimisation of (7) over a corresponding space. It leads to an approximation of the CE mean filter (11) for details see [11,10].

Numerical example of existing filters
Q|Z (y) PDF of error PDF of prior , and the probability measures P Ξ and P Θ with standard normal distribution N (0, 1). The random variables Q ∼ N (0, 1) with Q(ξ) = ξ and E ∼ N (0, 0.4) with E(θ) = 0.4 −1/2 θ determine the normal distribution of the parameter q = Q(ξ) ∈ Q and of the error. The correct posterior mean Mean Q|Z (y) = Φ Q|Z (y) (solid line with dots) is calculated from the PDF by (10a), approximations to CE Φ k Q|Z (y) (dashed lines) by the minimisation problem (7) using polynomial approximation of order k.
One can observe this from a simple one-dimensional example that is visualised in Figure 1 and described in detail in the caption of the figure. This simple one-dimensional example, which allows the exact expression of corresponding values and functions without additional approximation, depicts a big deficiency of some approaches. Particularly, the function Mean Q|Z (y) = Φ Q|Z predicts, from the PDF in (10a), correctly the mean of the conditional distribution for various observations y; in reality, there is only one observationŷ. Other functions Φ k Q|Z only approximates the conditional mean. Not surprisingly the linear approximation corresponding to the Kalman filter does not predict the mean correctly. However, also the higher polynomial approximations Φ Q|Z (y) k are of no big advantage. Particularly, the observed valueŷ = −2.5 would provide a big underestimation of the mean while the valueŷ = −10 would provide an underestimation using the linear approximation (Kalman filter) but an overestimation for polynomials of order 5 and 15. Moreover the approximation of the posterior covariance ΦQ ⊗Q|Z (ŷ) with multivariate polynomials is very poor and usually fails to be even positive semi-definite.
To conclude, the presented filters can predict only the mean of the updated random variable when the conditional expectation is calculated accurately. However, the polynomial approximation typically leads to a big error. Therefore it fails to predict even the mean correctly. This will be overcome in the next section using conditioned expectation.

Conditioned expectation and its approximation
In accordance with the discussion in the previous section, the conditional expectation of Q, while observing Z, predicts only the mean correctly, i.e. the value Φ Q|Z (ŷ) that is called conditioned expectation (CdE). It is thus superfluous to calculate the whole optimal map Φ Q|Z : Y → Q of the conditional expectation (CE) when only its value atŷ is needed. The following theorem shows how to reformulate CE into CdE for a general random variable X : Ω → X , possibly a function of the parameter Q.
Theorem 3 (Conditioned expectation). Let X ∈ L 1 (Ω, S, P; X ) be a random variable such that X(ξ, ·) is continuous and X a finite dimensional space. Let Z ∈ L 1 (Ω, S, P; Y) be a measurement (3) with an error E, which is bijective (one-to-one map) and has a continuous probability density function f E . Then the conditional expectation Φ X|Z at the point of observationŷ is equal to Let us make some remarks before the proof will be presented.

Remark 4.
Since the value f E (ŷ − Y (ξ)) is the likelihood, the conditional expectation equals the weighted average of the random variable with weights being the likelihood (8).
Proof of Theorem 3. The proof is based on the approximation of a conditional expectation using simple functions along with a limiting process. Since one does not need the whole conditional expectation but only its value atŷ, the approximation of CE with a measurable map Φ δ : Y → X using simple functions is convenient because of their localisation effect. Moreover, simple functions are dense on L 1 as well as L ∞ . Starting with Q = R for simplicity, the approximation with disjoint sets A i covering Y is considered. Indeed, there exists an index j such that the observationŷ is contained in A j . For notational simplicity we will denote this space A j as Aŷ =ŷ + A δ , where A δ is some δ-neighbourhood of 0; particularly, some neighbourhood from Borel σ-algebra B Y is considered, e.g. a subset M i=1 (−δ/2, δ/2) centred atŷ in Y.
The approximation of CE with simple functions at the observationŷ equals the particular coefficient α i that can be obtained from the Galerkin approximation of the orthogonality condition (22), i.e.
This is a fully decoupled (diagonal) linear system for α i when it is tested with the characteristic functions.
It can be simplified by integration over the support of the random variable 1 Aŷ • Z, i.e. a set The orthogonality (16) leads to a simple approximation of the conditional expectation .
The conditional expectation can be then calculated as a limit for δ → 0, thanks to the density of simple functions. The integrals over Θ δ (ξ) can be averaged with the volume of A δ (i.e. with Lebesgue measure λ of A δ ) in both nominator and denominator. Then the limit in the nominator is calculated as The limit in the denominator is calculated as for X(ξ, θ) = 1 to obtain a likelihood f E [ŷ − Y (ξ)]. Both together yield the required formula (15) for X = R. Now, the general case for finite dimensional X will be proven. Taking v ∈ X one can apply the proof on the scalar-valued random variable X · v. The linearity of conditional expectation and of the integral leads to the required conditioned expectation .
The proof is finished since it holds for all v ∈ X .

Remark 5.
We note that the space L 1 (Ω, S, P; Q) of parameters is defined as a product space where Ξ is the original domain of the parameter Q while Θ is the domain for a random variable of error E. This means that the error is assumed to be independent of the parameters. The conditioned expectation presented here (15) is valid for a special also case when X(ξ, η) = Q(ξ), see Corollary 7, as well as for the more general random variable X, depending also on the error variable θ. This could be useful when estimating the whole distribution of the updated random variable and not only the mean.

The connection of conditioned expectation with probability densities
Here we discuss the connection with conditioned expectation (Theorem 3) and the evaluation of mean (10a) and covariance (10b) based on probability densities.

Lemma 8.
The mean (10a) and covariance (10b), obtained from Bayesian updating using probability densities, are equal to conditional expectations Proof. The calculation follows directly from the change of the variable formula in the integral 26. For clarity, the statements is explicitly written for a function X(Q) of Q where a push-forward measure P Q is defined as P Q (A) = P[Q −1 (A)] for A ∈ B Q , and the probability density f Q is defined with respect to the Lebesgue measure. The formula for the posterior mean is obtained with X(Q) = Q and for the posterior covariance with X(Q) = (Q − Φ Q|Z (ŷ)) ⊗ (Q − Φ Q|Z (ŷ)).

Numerical approximation of conditioned expectation
The conditioned expectation, stated in Theorem 3, has to be integrated over the domain Ξ. In practise, the integrals can be evaluated approximately using some numerical integration rule with integration points ξ i ∈ Ξ and corresponding integration weights w i ∈ R for i ∈ {1, 2, . . . , k}, which are expressed with respect to the probability measure P Ξ . As an alternative, the Monte Carlo method can be used with randomly chosen integration points, again w.r.t. P Ξ , and equal weight w i = 1 n . The natural question arises about the positive-definiteness of the approximated covariance. The positive answer is summarised in the lemma. We emphasise that the covariance computed with the linear Φ 1Q ⊗Q|Z (ŷ) or non-linear Φ k Q⊗Q|Z (ŷ) approximation to the CdE is not guaranteed to be positive semi-definite, and often it fails.

Lemma 9 (Covariance). Let ξ i be the points of a numerical integration with positive weights
Then the approximated covariance is positive semi-definite.
Proof. Let C ∈ Q ⊗ Q be an approximation to a covariance matrix, i.e.
. Then for every q ∈ Q, we can calculate Remark 10 (Parameters as PCE). In many cases the parameters Q are approximated in terms of polynomial chaos expansion (PCE), i.e. Q(ξ) = p k=1 Q k ψ k (ξ). In the numerical integration of CdE it requires only the additional evaluation of polynomials .

Remark 11 (Parameters as samples). Another option for parameters is when they are given as (Monte Carlo) samples
. In that case the samples are used for integral evaluation in (18).

Numerical examples
Numerical examples are presented here to confirm the theoretical findings. All the numerical values were calculated as exactly as possible to suppress any error stemming from the numerical integration. Particularly, we have utilised Gauss-Hermite quadrature of sufficiently high order, the QUADPACK library [6] for an automated integration, or the Monte-Carlo method to doublecheck the results.
The first example is exactly the same as in Figure 1 that compares the Kalman filter with higher-order polynomial approximations of conditional expectation. The results presented in  The second example, presented in Figure 3, is an example with a non-monotonous solution operator that leads to double peaks of the conditional distribution. Again, it clearly confirms that conditional mean and covariance are accurately predicted by conditioned expectation.
The last example focuses on a problem with two parameters and one measurement, i.e. when Y = R and q belongs to Q = R 2 and. This example is described in the caption of Figure 4, which shows a response surface of the forward problem. The likelihood function (8) and the posterior distribution (in terms of probability density (9)) are depicted in Figure 5. Particularly, it shows the conditional expectation and its approximations that predict or approximate the updated mean; the concrete values are summarised here One can notice the big difference between conditioned expectation Φ Q|Z (ŷ) predicting the mean accurately and its polynomial approximation. Particularly, the conditioned expectation predicts the covariance matrix      Figure 5: Bayesian updating for the two-parameter problem from Figure 4 with likelihood (left) and the probability density of the posterior distribution (9) (right), which also shows the accurate conditioned expectation Φ Q|Z (ŷ) of the mean and its polynomial approximations (14).

Conclusion
This paper is focused on the computation of conditional expectation (CE) which is the proper approach in inverse problems using Bayesian updating in terms of random variables. It is only necessary to evaluate the conditional expectation Φ Q|Z at the point of measurementŷ in order to obtain conditional probability distribution. Therefore, under the assumption that the observation is accompanied with an additive error, the CE is reformulated to express directly the value Φ Q|Z (ŷ), here called a conditioned expectation (CdE); the main result is stated in Theorem 3 and Corollary 7. The consequences and other outcomes are summarised as follows: • CdE predicts correctly a conditional mean (17a) and covariance (17b), which overcomes the linear approximation of CE used in the Kalman filter (13) or CE approximated with multivariate polynomials (14), see numerical example in Figure 1.
• Numerical implementation (18) of CdE is straight-forward and requires only a numerical integration over the domain of the prior random variable; the approximation of CE with polynomials requires also integration over the error domain.
• For a numerical integration scheme with positive weights, the approximated covariance matrix (19) using CdE is always positive semi-definite (Lemma 9).
• The connection between CdE and Bayesian updating in terms of probability measures (densities) is established (Lemma 8). It is essentially a change of variables.

A Conditional expectations, probabilities, and distributions
This section contains an abstract description of conditional expectation, conditional probability, and conditional distribution; the diagram of the spaces and maps can be found in (4). When it is useful for a better understanding, the proof or its outline is presented.

A.1 Conditional expectation
Definition 12. Let X ∈ L 1 (Ω, S, P; X ) be a random variable and Z ∈ L 1 (Ω, S, P; Y) be a measurement with Borel σ-algebra B on Y. Let be the σ-algebra generated by Z. Then the conditional expectation of X w.r.t. Z is a random vector

Remark 13. In many manuscripts, the conditional expectation E X|σ(Z)
: The required formula (21) for scalar valued X is given by the change of variable formula in integral in Lemma 20, i.e. ν(B) = B Φ X|Z (y)P Z ( dy) = Z −1 (B) Φ X|Z • Z(ω)P( dω). For a vector valued random variableX, the conditional expectation exists for the random variable X(ω) = X (ω), v X , where v is an arbitrary element from X . The rest can be shown using linearity of integrals and of the conditional expectation.

Lemma 15.
The conditional expectation is equivalent to the problem: Find Φ X|Z ∈ L 1 (Y, B, P Z ; X ) such that the following orthogonality condition holds Proof. Starting from the definition of CE (21), the integrals over the domain A ∈ σ(Z) can be rewritten with the help of characteristic function Since the values of the integral are in X , the scalar product has to be used with the help of an arbitrary vector v ∈ X to obtain the duality with the function v1 A • Z ∈ L ∞ (Ω, σ(Z), P; X ). The final orthogonality condition (22) can be obtained by linearity and density of simple functions in L ∞ (Ω, B, P Z ; X ).

Remark 16.
For X ∈ L 2 (Ω, S, P; X ) the CE is from a smaller space Φ X|Z ∈ L 2 (Y, B Y , P Z ; X ) and the orthogonality condition changes from the duality into the scalar product

Moreover the conditional expectation can be formulated variationally as a minimisation problem
(Ω,S,P;X ) .

A.2 Conditional probability and distribution
Here we discuss the connection between conditional expectation, conditional probability, and its density. In the previous section we introduced the conditional expectation Φ X|Z of a random variable  Proposition 17 (Connection of conditional probabilities to probability densities). Let Q ∈ L 1 (Ω, S, P; Q) be a parameter RV and Z ∈ L 1 (Ω, S, P; Y) be a measurement RV, and P Q,Z be a probability measure on then the conditional probabilities can be expressed in terms of probability densities Proof. The proof that is provided in [13, Section 3.2, Proposition 7] for cumulative distribution functions is formulated here in terms of conditional probabilities. The definition of the joint probability P Q×Z on Q × Y can be rewritten using conditional expectation w.r.t. Z, for A ∈ B Q and B ∈ B Y Using probability density also for P Q,Z , we obtain the relation in PDF A×B f Q,Z (q, y) dq dy = A×B f Q|Z (q|y)f Z (y) dq dy.
for arbitrary sets A ∈ B Q and B ∈ B Y . Therefore, the similar relation holds for the arguments f Q,Z (q, y) = f Q|Z (q|y)f Z (y) for almost all (q, y) ∈ Q × Y.
Analogical result can be obtained with a change of random variables f Q,Z (q, y) = f Z|Q (y|q)f Q (q), from which the conditional probability density can be deduced To finish the proof, we will show that Q f Z|Q (y|q)f Q (q) dq = f Z (y). Indeed Next lemma is specialised for a measurement with additive error which is described in more detail in Section 2.2. Proof. We start from the conditional probability distribution which is defined as In fact the integral over the intersection of preimages Q −1 (A) ∩ Z −1 (B) is needed. We will use that Q does not depend on the whole domain Ω = Ξ × Θ but only on ξ ∈ Ξ and error E only on θ ∈ Θ. Particularly the preimages can be expressed as