{'fraction_non_alphanumeric': 0.03359039003410945, 'fraction_numerical': 0.010677739878392408, 'mean_word_length': 4.455906148867314, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Error bounds for some approximate posterior measures in Bayesian inference

In certain applications involving the solution of a Bayesian inverse problem, it may not be possible or desirable to evaluate the full posterior, e.g. due to the high computational cost of doing so. This problem motivates the use of approximate posteriors that arise from approximating the data misfit or forward model. We review some error bounds for random and deterministic approximate posteriors that arise when the approximate data misfits and approximate forward models are random.


Introduction
An inverse problem consists of recovering an unknown parameter u that belongs to a possibly infinite-dimensional space U from noisy data y of the form where Y is the 'data space', G : U → Y is a known 'forward operator', and η is a random variable. In many problems of interest, the parameter space U is a subset of an infinite-dimensional Banach space, the data space Y is often taken to be R d for some possibly large d ∈ N, and η is assumed to be Gaussian.
One of the main difficulties with inverse problems is that they often do not satisfy Hadamard's definition of well-posedness. To circumvent this difficulty, one may use the Bayesian approach, in which one incorporates information about the unknown u from existing data and from new data in the 'prior' probability measure µ 0 on U and in the 'data misfit' Φ : Y × U → R respectively. If η ∈ R d in (1) is distributed according to the normal distribution N(0, Γ) with positive definite Γ ∈ R d×d , then Φ(y, u) := 1 2 Γ −1/2 (y − G(u)) 2 .
The posterior µ y describes the distribution of the unknown u, conditioned upon the data y. By imposing conditions jointly upon Φ and µ 0 , one can show that the Bayesian solution µ y to the inverse problem depends continuously on the data, and one can prove the well-posedness of the Bayesian inverse problem; see [1]. For simplicity, we shall assume that the data y is given and fixed, and omit the dependence of the posterior, data misfit, and normalisation constant Z on y.
One challenge with solving Bayesian inverse problems in practice is that it is often not possible or desirable to evaluate the data misfit Φ(u) exactly. It then becomes necessary to find approximations Φ N of the true data misfit Φ that can be computed more efficiently, such that for sufficiently large values of N, inference using the approximate misfit Φ N effectively approximates inference using the true misfit Φ. Thus, one needs to identify conditions on Φ N such that two criteria are fulfilled: first, an approximate posterior measure µ N defined by exists and is well-defined; and second, the approximate posterior µ N provides an increasingly good approximation of the true posterior µ as the approximation parameter N increases. In this paper, we review results from [2] that guarantee well-definedness of µ N and establish error bounds for µ N in terms of error bounds for Φ N . In recent years, randomised numerical methods have been developed in order to overcome limitations of their deterministic counterparts. The field of probabilistic numerical methods [3] injects randomness into existing deterministic solvers for differential equations in order to model the uncertainty due to unresolved subgridscale dynamics. Random approximations of the forward model have been applied for forward uncertainty propagation in a range of applications; see e.g. [4,5].
Randomisation has been shown to yield gains in computational efficiency. Results from [6, Section 5.7] showed a reduction by a factor of almost 25 in the CPU time needed for generating an independent sample with the Metropolis-Hastings algorithm, while in [7], a multilevel Markov Chain Monte Carlo method uses ran-domisation in the form of control variates for variance reduction. Stochastic programming ideas were used for more efficient posterior sampling in [8]. The results we describe provide theoretical support for the use of randomisation in Bayesian inference, and extend the pioneering results from [9], which concerned Gaussian process approximations of data misfits and forward models.
To motivate the use of random approximate misfits, consider the following example: Let X be any In [10], the misfit Φ N above was used to obtain computational cost savings when solving inverse problems associated to PDE boundary value problems. The results we present below can be specialised to the case of X with bounded support [2, Proposition 4.1]. For example, we can use the ℓ-sparse distribution for some 0 ≤ ℓ < 1; for ℓ = 0, this is the Rademacher distribution. Similar ideas have been applied for full waveform inversion in seismic tomography [11], for example.

Error bounds for approximate posteriors
In what follows, we shall assume that the parameter space U admits a Borel σalgebra, and we shall denote by M 1 (U) the set of Borel probability measures on U.
Recall that the Hellinger metric d H : where π ∈ M 1 (U) is any measure such that µ and ν are both absolutely continuous with respect to π. It is known that d H does not depend on the choice of π.

Error bounds for random approximate posteriors
We first present error bounds on random approximate posteriors µ N associated to random misfits Φ N , where N ∈ N. That is, given a probability space (Ω, F , P), we shall view a random misfit as a measurable function Φ N : Ω × U → R. Furthermore, we shall assume that the randomness associated to the approximate misfit Φ N is independent of the randomness associated to the unknown parameter u. In what follows, ν N denotes a probability measure on Ω with the property that the distribution of the random function Φ N is given by ν N ⊗ µ 0 . Given (3) and (4), a natural question is to establish an appropriate bound on the Hellinger distance between the true posterior µ and the approximate posterior µ N in terms of some norm of the error between the true misfit Φ and the approximate misfit Φ N . We emphasise that the approximate posterior µ N in (4) is random in the sense that it depends on ω, since the approximate misfit Φ N depends on ω. Therefore, the Hellinger distance d H (µ, µ N ) will depend on ω as well. To describe such a bound, we shall take the expectation of the Hellinger distance with respect to ν N , and let for any Borel-measurable function f : R → R and q ∈ [1, ∞). We define the quantity With these preparations, we present the following theorem, which was given in [2, Theorem 3.2].
Theorem 1 (Error bound for random approximate posterior) Let (q 1 , q ′ 1 ) and (q 2 , q ′ 2 ) be pairs of Hölder conjugate exponents, and let D 1 , D 2 be positive scalars that depend only on q 1 and q 2 . Suppose the following conditions hold: .
Theorem 1 provides a bound on the mean square Hellinger distance between the true posterior µ and the random approximate posterior µ N , in terms of an appropriate norm of the error Φ − Φ N . The bound (5) implies that the negative tails of both Φ and Φ N must decay exponentially quickly with respect to the ν N ⊗ µ 0 -measure, and is satisfied, for example, when both Φ and Φ N are bounded from below. Since it follows that the constraint imposed on the misfit Φ N by (6) is that exp(−Φ N ) should be neither too concentrated nor too broad. Together, conditions (5) and (6) ensure that the random approximate posterior µ N exists, is well-defined, and satisfies the desired bound on the mean square Hellinger distance with respect to the true posterior µ.
An alternative way to generate an approximate posterior measure given a random approximate misfit is to compute a marginal approximate posterior µ M N , defined by Note that, since we have taken expectations with respect to ν N , the marginal approximate posterior does not depend on ω, and is in this sense deterministic. The following theorem was given in [2, Theorem 3.1].
Theorem 2 (Error bound for marginal approximate posterior) Let (p 1 , p ′ 1 ), (p 2 , p ′ 2 ), and (p 3 , p ′ 3 ) be Hölder conjugate exponent pairs, and suppose there exist finite, positive scalars C 1 , C 2 , and C 3 that depend only on p 1 , p 2 , and p 3 , such that the following conditions hold: Then there exists C > 0 that does not depend on N such that The bounds in (10) ensure that the denominator in (7) is strictly positive and finite. Thus, these bounds play a fundamental role in ensuring that the marginal approximate posterior exists and is well-defined. The bound in (9) reiterates the bound (5), modulo the 1 2 factor, and thus serves a similar purpose as (5). The bound in (8) serves a similar purpose as (6). However, the minimum operator implies that it is not necessary for both Φ and Φ N to be well-behaved.
The following result is a corollary of Theorem 1, Theorem 2, and [2, Lemma 3.5]. The main idea is to specify sufficient conditions for the hypotheses of both Theorem 1 and Theorem 2 to hold.

Corollary 1 (Joint conditions for error bounds on both approximate posteriors)
Suppose the following conditions are satisfied: and where . If in addition to conditions then the constants C and D in (11) and (12) do not depend on N.
Condition (i) amounts to a common uniform lower bound on all the misfits, both the true misfit and the collection of random approximate misfits, and thus plays a role in ensuring that (5) and (9) are satisfied. Condition (ii) makes precise the assumption that Φ N approximates Φ in the L 1 ν N ⊗µ 0 topology, which is a necessary condition for ensuring that the right-hand sides of the conclusions of Theorem 1 and Theorem 2 are finite. Condition (iii) describes an exponential integrability condition on the random approximate misfits and ensures that (6) and (8) are satisfied. Thus the additional condition amounts to a uniform exponential integrability condition over all sufficiently large values of N.
Remark 1 Neither Theorem 1 nor Theorem 2 require boundedness from below of either Φ or the Φ N . However, the negative tails of both Φ and Φ N must decay exponentially quickly at a sufficiently high rate, as specified by (9) and (5) respectively.

Error bounds for random forward models
Next, we consider approximate posterior measures that arise as a result of approximating the forward model G in (1). For simplicity, we shall consider only the case when the data misfit Φ and forward model G are related via the quadratic potential (2). In particular, this means that if G N : U → Y is an approximation of the true forward model G, then the resulting approximate data misfit is given by The following theorem is a nonasymptotic reformulation of [2, Theorem 3.9 (b)].

Theorem 3 (Error bounds for approximate posteriors) Suppose there exists
for C, D > 0 that do not depend on N.
The theorem can be rewritten so that, instead of imposing a uniform exponential integrability condition on the approximate quadratic potentials Φ N , one instead imposes an exponential integrability condition on the true data misfit Φ; see [2, Theorem 3.9 (a)]. An additional hypothesis in this case is that the expectations of the approximate data misfit functions are ν N -almost surely bounded, in the sense that for some C 4 ∈ R that does not depend on N.

Conclusions and directions for future work
This paper has reviewed the main error bounds of [2] concerning deterministic and random approximate posteriors that arise when performing Bayesian inference with random approximate data misfits or random forward models. The error bounds on the approximate posterior measures are given with respect to the Hellinger metric on the space of Borel probability measures M 1 (U). Given a fixed prior measure µ 0 , these error bounds describe -with specific exponents of integrability and problemdependent constants -the local or global Lipschitz continuity of the map that takes a data misfit as input and produces the corresponding posterior measure as output. Aside from the regularity assumptions made on the random approximations, the error bounds shown above make no structural assumptions on the approximations used. For example, we do not assume that the random approximations involve Gaussian random variables, or random variables with bounded support. Recent work has highlighted the importance of considering other metrics on M 1 (U), and also of proving well-posedness of the solution of a Bayesian inverse problem by establishing continuous (instead of Lipschitz continuous) dependence on either the data, prior, or data misfit. The well-posedness of Bayesian inverse problems in the sense of continuous dependence with respect to the data of the posterior for given prior and data misfit was established in [12]. Local Lipschitz continuity with respect to deterministic perturbations in the prior or data misfit was shown in [13]. In both [12,13], continuity is with respect to the topologies induced by the total variation metric, by Wasserstein p-metrics, or by the Kullback-Leibler divergence.
A key assumption made in [13] when establishing local Lipschitz continuity for a fixed prior µ 0 with respect to perturbations in the data misfit is that the deterministic perturbed data misfit is µ 0 -almost surely bounded from below. As highlighted in Remark 1, the analysis of [2] does not require that either the true data misfit or the random approximate log-likelihood are µ 0 -almost surely bounded from below. For future work, we aim to establish similar continuity results with respect to different metrics, as demonstrated in [12,13], but at the same level of generality of [2].