{'fraction_non_alphanumeric': 0.056087623683792796, 'fraction_numerical': 0.02444573786972856, 'mean_word_length': 3.7992635855764347, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 20, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Convergence rate bounds for iterative random functions using one-shot coupling

One-shot coupling is a method of bounding the convergence rate between two copies of a Markov chain in total variation distance, which was first introduced by Roberts and Rosenthal and generalized by Madras and Sezer. The method is divided into two parts: the contraction phase, when the chains converge in expected distance and the coalescing phase, which occurs at the last iteration, when there is an attempt to couple. One-shot coupling does not require the use of any exogenous variables like a drift function or a minorization constant. In this paper, we summarize the one-shot coupling method into the One-Shot Coupling Theorem. We then apply the theorem to two families of Markov chains: the random functional autoregressive process and the autoregressive conditional heteroscedastic (ARCH) process. We provide multiple examples of how the theorem can be used on various models including ones in high dimensions. These examples illustrate how the theorem's conditions can be verified in a straightforward way. The one-shot coupling method appears to generate tight geometric convergence rate bounds.

convergence rate for Markov chains defined in discrete state spaces [43,40,38]. On the other hand, despite the major developments made in bounding Markov chains in continuous state space, many applications of continuous state space Markov chains do not have established convergence rate bounds. For example, convergence rate bounds applied to Markov chain Monte Carlo (MCMC) models, which are useful for deciding the size of the burn-in period [16,11], do not have known upper bounds on their convergence rate [11]. Users need to rely on ad-hoc convergence diagnostics (e.g., [10]), which offer no guarantees.
The minorization condition is satisfied for a Markov chain {X n } n≥1 under the following circumstances: there exists a small set K, a probability measure Q and a positive number > 0 such that P (· | X n = x) ≥ Q(·) for x ∈ K. The drift condition is satisfied if there exists a positive function V , and constants α > 1 such that [28,35]. Bounds generated using the drift and minorization conditions have been applied to a wide array of problems such as [39,47,16].
Despite the widespread use of bounds generated by the drift and minorization conditions, there are drawbacks.
First, it can be a challenge to identify a small set K and drift function V [27]. Second, it is shown in [32] based on results from [19] that bounds that use the minorization condition do not scale well in high dimensions.
Alternatively, methods for finding Markov chain convergence rate bounds on the Wasserstein distance have been shown to scale well in high dimensions [8,32], so bounding the total variation distance by first bounding the Wasserstein distance is a common technique used in the literature [32,27,21].
One-shot coupling, which bounds the Wasserstein distance as an intermediate step [27], provides an upper bound on the convergence rate in total variation distance of a Markov chain. This method does not need to identify any exogenous sets or functions, like the drift and minorization conditions. Further, the one-shot coupling method has already been shown to scale well in certain high dimensional examples [37,30] and will be shown in this paper to scale well in high dimensions for the Bayesian regression Gibbs sampler (Example 4.2) and the Bayesian location Gibbs sampler (Example 4.3).
The one-shot coupling method described in [37] works by first converging the expected distance between two copies of a Markov chain. At the last iteration, the probability of coupling is evaluated when the expected distance between the copies is small. This contrasts with the drift and minorization technique, which attempts to couple the two Markov chain copies every time they enter some fixed small set K.
In this paper, we introduce the One-Shot Coupling Theorem in Section 3, which aims to summarize the method defined in [37] and [27] for straightforward applications. The One-Shot Coupling Theorem is used as the foundation for bounding the convergence rate for all of the examples in this paper, which can be partitioned into two families: the random functional autoregressive process and the ARCH process. In Section 4 we introduce the Sideways Theorem 4.1, which is new and is an application of the One-Shot Coupling Theorem. We apply it to various examples of random functional autoregressive processes (definition 4.1). In Section 5 we provide convergence rate bounds using the One-Shot Coupling Theorem to various ARCH processes (definition 5.1).
Proofs for the theorems presented in this paper are found in the appendix, Section 7. The code used to generate all of the tables and calculations can be found on github.com/sixter/OneShotCoupling.

Background and notation
Let {X n } n≥1 and {X n } n≥1 be two copies of the Markov chain over the state space X and define L(X n ) to be the distribution of the random variable X n . We define π to be the stationary distribution of the Markov chain.

Total variation distance
We are interested in measuring the distance between the distribution of two Markov chains. To measure this we use the total variation metric. Definition 2.1 (Total variation distance). The total variation distance between the laws of two random variables, X and X , defined on the state space X is where L(X) represents the distribution of the random variable X and A is a measurable set.
For random variables, X, X ∈ R with defined density functions f X , f X and reference measure λ, Total variation distance has natural probability interpretations. It is the maximum difference in probabilities of an event. It is the error in an expected bounded loss function when a given measure is used as a proxy for another [13]. Finally, it can be seen as the percentage of samples of L(X) which cannot be regarded as samples from L(X ) (Proposition 3(g) [35]).
Historically, total variation distance was the common metric for measuring Markov chain convergence rates [35,28,22,16] and hence, there is a rich literature of attributes that can be deduced from finding convergence rates in total variation. For example, mixing times in total variation distance can be used to determine whether the Markov chain is asymptotically uncorrelated (Theorem 2 of [22]), uniformly integrable (Theorem 3 of [22]), whether the central limit theorem (CLT) applies (Theorem 9 of [22] or Section 5.2 of [35]), or whether it is convergent based on the total variation mixing times of another Markov chain (Theorem 8 of [9]).
The following are properties of total variation, which will be used in conjunction with the One-Shot Coupling Proposition 2.1. Let X, X ∈ X be two random variables and let g : X → Y be an invertible and measurable function. Then, The proof is in Section 7.1.
In general, for a measurable (not necessarily invertible) function g, g −1 (f (B)) ⊂ B, so the third equality in the proof becomes ≤ and L(g(X)) − L(g(X )) ≤ L(X) − L(X ) Proposition 2.2 states that the total variation distance between two random variables is bounded above by the expected value of the conditional random variable.
Proposition 2.2. Let X, X be two random variables with corresponding σ-field B and Y ∈ Y be some related random variable. Then The proof is in Section 7.1.
Proposition 2.3 states that the convergence rate of a Markov chain in R d with independent coordinates is d times the maximum coordinate-wise convergence rate. This proposition is an application of inequality 1.2 of [34].
Markov chain such that each coordinate is independent of the other coordinates, X i,n ⊥ ⊥ X j,n , i = j. Further suppose that for two copies of the Markov chain { X n } n≥1 and { X n } n≥1 , max 1≤i≤d L(X i,n ) − L(X i,n ) ≤ Ar n for some A ∈ R + and r ∈ (0, 1). Then, The proof is in Section 7.1.
In this paper, we establish convergence bounds for Markov chains that are geometrically ergodic in total variation distance.
Definition 2.2 (Geometric ergodicity). Let {X n } n≥1 be a Markov chain with a stationary distribution π. The Markov chain is geometrically ergodic if there exists a ρ < 1 and a function M (x) < ∞, π-a.e. such that for The geometric rate of convergence for X n is defined as ρ * = inf{ρ : equation 4 holds}.
Proposition 4 of [32] states that for any sequence of drift and minorization conditions, the geometric convergence rate ρ established by the Rosenthal bound (Theorem 12 of [42]) will increase at an exponential rate for the autoregressive normal process in R d as the dimension d → ∞. This finding suggests that convergence bounds that use the drift and minorization condition do not scale well in dimension (see Lemma 3 and discussion in [32]

Wasserstein distance
Let X, X ∈ R be two random variables equipped with the Euclidean distance. The Wasserstein distance is defined as follows, In comparison to total variation distance, there is not as much literature dedicated to Markov properties that can be derived from the convergence in Wasserstein distance. However, this literature is growing. For example Jin and Tan provide sufficient conditions in [20] for the CLT based on convergence in Wasserstein distance (see Theorems 9 and 10).

Coupling
Total variation can also be defined in terms of the coupling characterization [13], The total variation metric measures the distance between two distributions, but is invariant to how these measures are jointly distributed. For example, let X ∼ N (0, 1) and X ∼ N (1, 1) be two random variables.
Regardless of whether X and X are highly dependent, for example if X = X +1 or if X, X are independent, their total variation distance would be the same. The Nummelin splitting technique makes use of this by constructing alternative random variables, Y and Y , such that the marginal distributions are the same L(X) = L(Y ), L(X ) = L(Y ), and the probability that they are unequal is minimised. This technique was first shown in [29].
See [40] or [28] for an explanation. Finally, note that the theory on maximal coupling guarantees that there exists alternative random variables Y, Y as defined above, such that L(X) − L(X ) = P (Y = Y ) [4].
Coupling techniques are widely used to calculate total variation upper bounds on Markov chains [38,42,40,35,39,48]. Let {X n } n≥1 and {X n } n≥1 be two copies of a Markov chain. If we want to use the coupling characterization for finding an upper bound on the total variation distance, we must also make sure that the faithful coupling condition holds (see Section 2 of [41]). That is, for any measurable set A ∈ X , If the faithful coupling condition holds, then calculating the probability that two chains are unequal at iteration n can be interpreted as the probability that the two chains have not yet coupled by iteration n. This is because once the two Markov chains couple, they can be structured so that they are equal forever and so P (X n = X n ) = P (T ≥ n) where T = min{k : X k = X k } (Theorem 1 of [41]). If a minorization condition holds on the Markov chain, then the faithful coupling condition also holds. For one-shot coupling, we do not need faithful coupling, because we only try to couple the chains at the last iteration.

One-Shot Coupling
One-shot coupling is an alternative way of applying coupling methods to bound the total variation of two copies of a Markov chain. To apply one-shot coupling, we define a Markov chain in terms of iterated random functions [6]. That is, define a family of random functions {g θ : θ ∈ Θ Θ Θ} such that θ n = {θ 1,n , . . . , θ d,n } is a random vector and X n = g θn (X n−1 ) The nth iteration of the Markov chain can be written in terms of X 0 = x as follows, Summarizing Section 3 of [37], to find an upper bound on the total variation distance between X N and X N = g θ N (X N −1 ) we do the following.
1. Contraction phase: For n < N , set θ n = θ n so that the two chains get 'closer' together.

2.
Coalescing phase: For n = N , we set all but one coordinate of θ n and θ n to equality and attempt to couple X n and X n . That is, specify coordinate j ∈ {1, . . . , d} and set θ i,n = θ i,n for all i = j. Then we attempt to jointly choose θ j,n and θ j,n , such that g (θ1,n,...,θj,n,...,θ d,n ) (X n−1 ) = g (θ1,n,...,θ j,n ,...,θ d,n ) (X n−1 ) The method used in the contraction phase is also known as the common random number method and is discussed in detail in Section 2.3.1 of [18]. The contraction phase can also be used to directly generate upper bounds in Wasserstein distance [18,31,12] (it is also used to generate bounds on other types of distances like Monge-Kantorovich or Prokhorov [18]).
The one-shot coupling method has been applied over a variety of specific examples, namely, a nested gamma model in [24], an image restoration model in [23], and a random walk on the unit sphere in [30].
The contraction and coalescing phase described above is how the one-shot coupling method was first defined in [37]. The following theorem summarizes the above method and serves as a general outline for bounding the total variation distance between two Markov chains. The coalescing condition below does not specify how the two chains will couple, unlike the method described above.
Theorem 3.1 (One-Shot Coupling Theorem). Let {X n } n≥1 , {X n } n≥1 be two copies of a Markov chain such that X n = g θn (X n−1 ) and X n = g θ n (X n−1 ), where (θ n , θ n ) n≥1 are independent random variables with respect to n and the marginal distribution of θ n , θ n ∼ D, for some distribution D. Suppose that the following two conditions hold for some non-negative integer n 0 .
1. Contraction condition: There exists a D ∈ (0, 1) such that for any n ≥ n 0 when θ n+1 = θ n+1 ∼ D 2. Coalescing condition: There exists a C > 0 such that for any n ≥ n 0 Then the total variation distance between the two Markov chains at iteration n ≥ n 0 is Proof of the One-Shot Coupling Theorem 3.1. Fix n ≥ n 0 . We are interested in finding an upper bound on L(X n ) − L(X n ) . To do so, we first generate alternative random variables, Y n , Y n such that 3. for m = n: θ m , θ m ∼ D with an arbitrary joint distribution and Next we find an upper bound on the total variation distance between Y n and Y n . By the contraction condition By the coalescing condition, Finally since Y n d = X n and Y n d = X n , If L(X n ) has a density function with respect to X n−1 = x, f (x, z), then Theorem 3.1 can be proven with Wasserstein distance as an intermediary using the following lemma.
If the contraction condition holds, then for n ≥ n 0 , W (L(X n−1 ), and the proof of Theorem 3.1 directly follows.
In most cases n 0 = 0. See the GARCH Example 5.3 for an alternative case, n 0 = 1.
In general, the contraction condition can be weakened. Theorem 1.1 of [6] provides sufficient conditions to guarantee the existence of D as defined in the above theorem. The conditions in Theorem 1 of [46], which are called local contractivity and are weaker, could also replace the contraction condition in the above theorem.
To bound the total variation between a Markov chain, {X n } n≥1 , and the corresponding stationary distribution, π, we set X 0 ∼ π. This implies that X n ∼ π and L(X and n 0 are satisfied according to the conditions above.
To find an upper bound on E X∞∼π [|X n0 − X ∞ |] we use the following Lemma 3.3, which uses a drift condition to bound the expected distance between the stationary distribution of a Markov chain and an initial value.
The expected distance between X 0 and X ∞ ∼ π is bounded above as follows,

Random-functional autoregressive processes
The following section proposes the Sideways Theorem to generate upper bounds on the total variation distance for random-functional autoregressive processes.
2. Attributes of the conditional density θ 2,n | θ 1,n : The conditional density of θ 2,n | θ 1,n (a) is bounded above: There exists a K > 0 such that for all (θ 1,n , θ 2,n ) ∈ R 2 , the conditional density function of θ 2,n is bounded above by K, f θ2,n (θ 2,n | θ 1,n ) ≤ K.  Then an upper bound on the geometric rate of convergence of the Markov chain is D and the total variation distance between the two copies of the Markov chain, X n , X n , is bounded above as follows, The attributes of the conditional density of θ 2,n | θ 1,n serve to prove, by integrating along the y-axis or flipping the density sideways, that the coalescing condition is satisfied. To prove the Sideways Theorem, we show that the contraction and coalescing conditions are satisfied and then apply the One-Shot Coupling Theorem 3.1.
Lemma 4.2 (Coalescing condition). If the density of θ 2,n |θ 1,n for any θ 1,n is (1) bounded above, (2) has at most M local extrema points that are at most L distance apart, and (3) is continuous then for n ≥ 0, Proof of Theorem 4.1. The following shows that the contraction condition holds for D ∈ (0, 1) and n ≥ 0, by contraction condition Lemma 4.2, which can be applied when condition 2 is satisfied (attributes of the conditional density of θ 2,n | θ 1,n ), shows that the coalescing condition holds. By the One-Shot Coupling Theorem 3.1, the total variation distance between two copies of the process can be bounded above using equation 5.
In [14], it is shown that when the function g is deterministic (g is a function of X n−1 only and not θ 1,n ) and given the same assumptions on θ 2,n , the upper bound on the geometric rate of convergence is D (see Corollary   8 and Example 9 of [14]). This matches the results from our theorem.
Note that the Sideways Theorem 4.1 provides an upper bound on total variation distance, but does not imply the existence of a stationary distribution for the Markov chain. To develop the intuition for this, first note that convergence in total variation distance implies convergence in distribution [13]. Suppose that L(X n ), L(X n ) have distribution functions, F n , F n , then by Helly's Selection Theorem (see Lemma 11.1.8 of [38]), a right continuous function F exists such that F n → F and F n → F pointwise. However, the function F may not necessarily be a distribution function. This is an illustration of why a stationary distribution may not exist. 1 2 X n−1 + n and θ 2,n = Z n . It is clear how the Sideways Theorem 4.1 could generate a geometric convergence bound over two iterations of the process if E[X 0 − X 0 ] < ∞, but X n , X n → ∞ almost surely and so there is no stationary distribution. See [46] for more information on sufficient conditions for stationarity.

An example of a non-linear autoregressive process
Example 4.1 (Non-linear autoregressive process). This example is discussed in Section 4 of [31]. Let {X n } n≥1 be a Markov chain such that where {Z n } n≥1 ∼ N (0, 1) are independent and identically distributed (i.i.d.) random variables. In [31], it is assumed that {Z n } n≥1 are i.i.d. random variables with a mean of 0 and a variance of 1.
) and so sup x∈R g (x) = 1. This cannot be used. Instead, we can find a value for D in terms of the second iteration. That is, The value of D as defined above can be written as The proof can be found in Section 7.3.1.
Using simulation, we can deduce that D 2 ≈ 0.813 2 = 0.661, which closely matches the geometric convergence rate found in [31] for the Wasserstein distance of D = 0.814.
Using the Sideways Theorem 4.1 notation, K = 1 √ 2π and M = 1. An upper bound on the total variation distance is Thus if X 0 = 1 and X 0 = 2, then after 20 iterations, the total variation distance between the two processes will be less than 0.01.

Random-coefficient autoregressive models
Since all of the conditions in Theorem 4.1 are satisfied, equation 5 holds.

Bayesian regression Gibbs sampler
Example 4.2 (Bayesian regression Gibbs sampler). Suppose we have the following observed data Y ∈ R k and for unknown parameters β ∈ R p , σ 2 ∈ R. Suppose we apply the prior distributions on the unknown parameters, The Bayesian regression Gibbs sampler is based on the conditional posterior distributions of β n , σ 2 n and is defined as follows.
The following theorem gives an upper bound on the convergence rate of the Bayesian regression Gibbs sampler.
In Theorem 3.1 of [33], it was shown than for the equivalent example and some 0 < M 1 ≤ M 2 , which are not specified, This means that the bound derived from the Corollary 1 is sharp up to a constant. The primary difference between Theorem 3.1 in [33] and the bound in Theorem 4.4 is that the latter provides explicit values for the constant, M 2 and as a result, numerical upper bounds can be calculated.
Before proving the Theorem 4.4, we present some lemmas.
Lemma 4.5. The variable σ 2 n can be written as a random-coefficient autoregressive process, The proof can be found in 7.3.2. Note that Γ(α, β) represents the gamma distribution and Γ −1 (α, β) represents the inverse gamma distribution.
The proof can be found in 7.3.2.
Given the above lemmas, the proof of Theorem 4.4 is straightforward when the Sideways Theorem is applied.
where K is defined in Lemma 4.7. Lemma 4.5 implies the first inequality. The second inequality is a result of Corollary 1, which is satisfied because of the contraction condition (Lemma 4.6) and the properties of the conditional density θ 2,n | θ 1,n (Lemma 4.7).
Numerical Example 4.1 (Application of the Bayesian regression Gibbs sampler). Suppose that we are interested in evaluating the delay in getting a PhD (Y ), based on age, age squared, sex and whether the student has a child at home (X). For more information on this problem see [44,45]. We want to find the upper bound on the total variation distance for a Bayesian regression Gibbs sampler fitted to this model. In this case, there are 333 observed values (k = 333) and 4 covariates (p = 4). Using the notation from Theorem 4.4, K = 0.0682. Further suppose we are interested in evaluating the upper bound between two copies of the Markov chain X n , X n such that σ 2 0 = 1 and σ 2 0 = 1001. Then, After 3 iterations, the total variation distance between the two chains will be less than 0.01. is based on the conditional posterior distributions of µ, τ −1 , which are defined as follows.
The following theorem gives an upper bound on the convergence rate of the Bayesian location model Gibbs sampler.
This bound compares to the one derived in Section 6 of [37] which states that, Both bounds return the same geometric rate of convergence. However, the magnitude of constant K is difficult to compare against J 2 + 1 without knowing S. Note that the bound derived from Corollary 1 is calculated in a systematic way.
Before proving Theorem 4.8, we present some lemmas.
The proof can be found in 7.3.3.
The proof can be found in 7.3.3.
Given the above lemmas, the proof of Theorem 4.8 is straightforward when the Sideways Theorem is applied.
Proof of Theorem 4.8. Note that where K is defined in  The proof can be found in 7.3.3.
So by lemma 3.3, Combining this with Theorem 4.8, After 4 iterations, the total variation distance between the two chains will be less than 0.01. This bound compares to the bound derived in [37], which, combined with equation 10, states that L(µ n , τ −1 n )−L(µ n , τ −1 n ) ≤

Autoregressive normal process
Example 4.4 (Autoregressive normal process in R). Let {X n } n≥1 ∈ R be an autoregressive normal process.
Then for i.i.d. Z n ∼ N (0, 1), In this case θ 1,n = 1 2 and θ 2,n = 3 4 Z n . The density of θ 2,n is continuous and uni-modal and K = 2 3π . By Corollary 1, It is known that the geometric rate of convergence for the autoregressive normal process is 1/2 [32], so once again the Sideways Theorem 4.1 generates tight geometric convergence rates up to a constant.
When comparing the upper bound with the actual total variation distance, note that if X 0 = x 0 is known, Thus, the total variation distance between two copies of an autoregressive normal process X n , X n where the initial values are known, X 0 = x 0 and X 0 = x 0 , is as follows (see Section 2 of [37]),  11) when X n , X n are two copies of the autoregressive normal process (i.e., N (0, 1)) and x 0 = 0, x 0 = 1. In the following section we extend the above example to higher dimensions.

Processes in R d
Next we extend the autoregressive normal process as defined above to R d . To do so, we apply Proposition 2.3 to an autoregressive normal process in R d with independent coordinates, Example 4.5, and non-independent coordinates, Example 4.6.
Example 4.5 (Autoregressive normal process in R d with independent coordinates). Let { X n } n≥1 ∈ R d be an autoregressive normal process with independent coordinates. Then for i.i.d. Z n ∼ N ( 0, I d ), And if i = j, then Z i,n ⊥ ⊥ Z j,n . Further, X i,n = 1 2 X i,n−1 + 3 4 Z i,n for i ∈ {1, . . . , d} and so by Example 4.4, Since each coordinate is independent and bounded above by the same value, Proposition 2.3 implies that Again, it is known that the geometric rate of convergence for the autoregressive normal process in R d is 1/2 [32].
Finally, to apply numbers to this example, suppose that X n , X n ∈ R 100 and the initial values of this process are X 0 = (1, . . . , 1) and X 0 = (0, . . . , 0). The total variation distance would be bounded above with L( X n+1 ) − L( X n+1 ) ≤ 100 2 3π 1 2 n . This means that at 14 iterations the total variation distance would be less than 0.01.
The following example is a more general version of the above, where X n is a general auto regressive normal process in R d . X n = A X n−1 + W n Theorem 4.13. Suppose that A is a diagonalizable matrix. Then for two copies, X n , X n ∈ R d , of the autoregressive normal process defined in Example 4.6, where A = P DP −1 with D as the corresponding diagonal matrix, λ i is the ith eigenvalue of A and · 2 denotes the Frobenius norm. The proof can be found in 7.3.4, which uses a modified version of the Sideways Theorem.
Numerical Example 4.3 (Application of the autoregressive normal process in R d ). To apply numbers to this example, suppose that X n , X n ∈ R 100 are two copies of the following process X n = A X n + Z n , Z n ∼ N (0, A)

Autoregressive conditional heteroscedastic processes
In this section we look at bounding the total variation distance between two copies of an ARCH process.

Application to the LARCH model
Example 5.1 (Linear ARCH process). Let {X n } n≥1 ∈ R be a linear ARCH process. Then for i.i.d. Z n and β 0 , β 1 ∈ R X n = (β 0 + β 1 X n−1 )Z n See Section 7.3.3 of [7] for more details on this model.
The following theorem provides an upper bound on the convergence rate of two copies of a LARCH process.
• the density of log(Z 0 ) is bounded above, has at most M local maxima and minima, and is continuous.
Then, the process is geometrically ergodic if β 1 E[|Z 0 |] < 1 and an upper bound on the total variation distance between the two processes is, Where D = β 1 E[Z 0 ] Lemma 7.3.2 of [7] says that if β 1 E[|Z 0 |] < 1, then a stationary distribution exists. This theorem makes an even stronger assertion that under some additional assumptions, the process will also be geometrically ergodic with geometric convergence rate D = β 1 E[|Z 0 |] < 1.
Numerical Example 5.1. We find the convergence rate of Example 10.3.1 of [5], which is of the form, After 3 iterations the total variation distance is less than 0.01. In comparison, Figure 2 shows how the bound compares to a simulated estimate of the total variation distance for this process.

Application to the asymmetric ARCH model
Example 5.2 (Asymmetric ARCH process). Let {X n } n n ≥ 1 ∈ R be an asymmetric ARCH process. Then for Where a, b, c ∈ R. See Exercise 4.1 of [7] for more details on this process.
The following theorem provides an upper bound on the convergence rate of two copies of an asymmetric ARCH process.
Theorem 5.4. Let {X n } n≥1 ∈ R and {X n } n≥1 ∈ R be two copies of the asymmetric ARCH process defined in Example 5.2. Suppose further that the density of Z n is centred at 0 and is monotonically decreasing around zero (i.e., π(x) ≥ π(y) if |x| < |y|.). Then, the process is geometrically ergodic if |a|E[|Z 0 |] < 1 and an upper bound on the total variation distance between the two processes is Where D = |a|E[|Z 0 |] Exercise 4.1 part 1 of [7] states that the process has a stationary solution if D = |a|E[|Z 0 |] < 1. Theorem 5.4 shows that under certain additional assumptions on Z n the process will also be geometrically ergodic with a specified quantitative bound.
So, by iteration n = 7, the total variation will be less than 0.01.
In comparison, Figure 4 shows how the bound compares to a simulated estimate of the total variation distance for this process. GARCH(1,1) process). Let {X n } n≥1 ∈ R be a GARCH(1,1) process. Then for i.i.d. Z n X n = σ n Z n where for α, β, γ ∈ R,

Application to the GARCH(1,1) model
See Section 7.3.6 of [7] for more details on this model.
The following theorem provides an upper bound in total variation distance between two copies of the GARCH(1,1) process.
Theorem 5.7. Let {X n } n≥1 ∈ R and {X n } n≥1 ∈ R be two copies of the GARCH process defined in Example 5.3. Suppose that the density of Z n is centered at 0 and is monotonically decreasing around zero. Then, the process is geometrically ergodic if β 2 E[|Z 2 0 |] + γ 2 < 1. Further suppose that x 0 , x 0 , σ 2 0 , and σ 2 0 are known. Then an upper bound on the total variation distance between the two processes is Before proving Theorem 5.7, we present some lemmas.
Where D = √ 0.1266 + 0.7922 = √ 0.9188 By iteration 77 the total variation distance between the two processes is less than 0.01. In comparison, Figure   4 shows how the bound compares to a simulated estimate of the total variation distance for this process. The actual total variation distance appears to be much smaller than the upper bound.

Acknowledgement
We thank the referees for their many excellent comments and suggestions. First note that f −1 (B) = {f −1 (B) : B ∈ B} = A: By invertibility, The equality in equation 2 can then be proven as follows, Proof of Proposition 2.2.
Proof of Proposition 2.3. To prove this we use the concept of maximal coupling over the coordinates. By maximal coupling, for i ∈ {1, . . . , d} there exists random variables X M i,n , X M i,n such that X i,n d = X M i,n and X i,n d = X M i,n and (see Proposition 3g of [35] or Section 2 of [4]).
Further, there exists a unique product measure such that for any [3]). For the unique product measure, the following equality holds, And so by uniqueness, for A ∈ B d , P (X M n ∈ A) = P (X n ∈ A). By definition this means that X n d = X M n , which implies that ( X M n , X M n ) ∈ C( X n , X n ), the set of all couplings of X n , X n .
We now use X M n , X M n to prove equation 3.

Lemmas related to the Sideways Theorem
The following are lemmas and corresponding proofs and corollaries related to the Sideways Theorem (4.1).

Lemmas providing an upper bound on the integral difference between a function and a corresponding shift
The following lemmas are used in the proof of Lemma 4.2.
Lemma 7.1. For any invertible, continuous function f : R → R where the codomain is f (R) = (a, b) and ∆ > 0, Proof. Since f is invertible and continuous, it is strictly monotone (Lemma 3.8 if [15]). Assume that f is strictly increasing. The integral can be written as follows, If f is strictly decreasing apply the transform h(x) = a + b − f (x). The function h is a strictly increasing invertible function with codomain (a, b) and so using the previous result for increasing functions, Note that g (x) is continuous, invertible, an increasing function and the codomain is (a, b). By Lemma 7.1 for Further, for all x ∈ R, lim →0 g (x + ∆) − g (x) = f (x + ∆) − f (x) and so g (x + ∆) − g (x) converges pointwise to f (x + ∆) − f (x). Next, for 0 < < (c − d)/2, |g (x + ∆) − g (x)| < 2|b| and so the function g (x + ∆) − g (x) is uniformly bounded. The above statements allow us to apply the dominated convergence Theorem (Theorem 16.5 of [3]) and so If f is strictly decreasing apply the transform h(x) = a + b − f (x). The function h is a strictly increasing invertible function with codomain (a, b) and so using the previous result for increasing functions, 3. Let f : R → R be a continuous function with the following properties: • the codomain is (0, K) • (m 1 , m 2 , . . . , m M ) are the local maxima and minima points • lim x→∞ f (x) = 0 and lim x→−∞ f (x) = 0

Show that
Note that over the interval (m i , m i+1 ], the function f is either a strictly increasing or a strictly decreasing function.

Ii+1
Ii The last equality is a result of Lemma 7.2.
By similar reasoning it can be shown that Finally note that the intersection points partition R into M + 1 subsets and so Proof of Lemma 4.2. Set θ 1,n = θ 1,n . Define ∆ = g(θ 1,n , X n−1 ) − g(θ 1,n , X n−1 ) Let f Xn , f X n be the density functions for X n , X n , respectively and f θ2,n , f θ2,n+∆ be the density functions for θ 2,n , θ 2,n + ∆.

Proof of lemmas used in Theorem 4.4
To prove the first part of this theorem, we apply the de-initialization technique which shows how the convergence rate of a Markov chain can be bounded above by the convergence rate of a more simpler Markov chain that includes sufficient information on the Markov chain of interest. The concept of de-initialization and a proposition that bounds total variation is provided below.
Proposition 7.4 (Theorem 1 of [36]). Let {Y n } n≥1 be a de-initialization of {X n } n≥1 then for any two initial distributions X 0 ∼ µ and X 0 ∼ µ , Proof of Lemma 4.5. Note that β n =β + σ n−1 Z n , Z n ∼ N p (0, A −1 ) can be written as a random function of σ 2 n .
Substituting β n , σ 2 n can then be written as a random function of its previous value for independent Z 2 n ∼ χ 2 (p) We can rewrite σ 2 n = X n Y n σ 2 n−1 +Y n where X n ∼ Γ p 2 , C 2 and Y n ∼ Γ −1 k+p 2 , C 2 . Using the notation from the Sideways Theorem 4.1 θ 1,n = X n Y n and θ 2,n = Y n .
Since β n can be written as a random function of σ 2 n , and so σ 2 n is a de-initialization of (β n , σ 2 n ). By Proposition 7.4, We are thus interested in evaluating the convergence rate of σ 2 n to bound the convergence rate of (β n , σ 2 n ).
To interpret this in another way, if σ 2 n couples then the distribution of β n is the same for both iterations, so it is automatically coupled. An alternative proof can be made using the results from [26].
Proof of Lemma 4.6. By Lemma 4.5, θ 1,n = X n Y n and so, Proof of Lemma 4.7. Calculate the conditional density θ 2,n | θ 1,n We remove the subscript n on the random variables. Let X, Y be as described in Lemma 4.5. Since the random variables are independent, the joint density is the product of the densities.
is proportional to f θ1,θ2 (θ 1 , θ 2 ) and so we can derive the conditional density of θ 2 as follows, This is proportional to an inverse gamma distribution and so, θ 2 | θ 1 ∼ Γ −1 k+2p 2 , (θ 1 + 1)C/2 . Since the conditional density is an inverse gamma distribution, the number of modes is M = 1 and the density function is continuous.

Proof of lemmas used in Theorem 4.8
Proof of Lemma 4.9. The iteration τ −1 n+1 can be written as a function of its previous value, τ −1 Next we can rewrite, Since (µ n , τ −1 n ) can be written as a random function of τ −1 n , and τ −1 n is a de-initialization of (µ n , τ −1 n ). Further, by Proposition 7.4, To interpret this in another way, if τ n couples then the distribution of µ n is the same for both iterations, so it is automatically coupled. An alternative proof can be made using the results from [26].
Proof of Lemma 4.10. By Lemma 4.9, θ 1,n = X n Y n and so by Corollary 1 Proof of Lemma 4.11. To find M, K and show that the conditional density is continuous, we (a) show that θ 2 | θ 1 ∼ Γ −1 J−1 2 , (θ 1 + 1)S/2 , which directly implies that the conditional distribution is continuous and M = 1 and we (b) we find the value of K.
(a) Calculate the conditional density θ 2,n | θ 1,n For simplicity, we remove the subscript n on the random variables. Let X, Y be as described in Lemma 4.9. Since the random variables are independent, the joint density is the product of the densities.
Proof of lemma 3.4. By the property of stationary distribution, if σ 2 n−1 ∼ π then σ 2 n ∼ π and so the lemma follows from the following.  Proof of Theorem 4.13. This example uses a modified version of the Sideways Theorem 4.1 to find an upper bound on the convergence rate. We will also use Proposition 2.1, which states that the total variation between two random variables is equal to the total variation of any invertible transformation of the same two random variables.
Let X n , X n ∈ R 2 be two copies of the autoregressive normal process as defined in Example 4.6. Then for Z n ∼ N ( 0, I d ), We apply the one-shot coupling method to bound the total variation distance. For n < N set Z n = Z n .
Suppose X 0 , X 0 are known and define Decompose A = P DP −1 with D as the corresponding diagonal matrix, λ i is the ith eigenvalue of A and · 2 denotes the Frobenius norm. Then ∆ is bounded above as follows, Proof of Lemma 5.5. Let {X n } n≥1 ∈ R and {X n } n≥1 ∈ R be two copies of the asymmetric ARCH process.
For a fixed n ≥ 1, let Z n = Z n and so, Note that the derivative of f (x) = (ax + b) 2 + c 2 is and so, Thus, the geometric convergence rate is D = |a|E[|Z 0 |].
Proof of Lemma 5.6. Let {X n } n≥1 ∈ R and {X n } n≥1 ∈ R be two copies of the asymmetric ARCH process.
For n ≥ 1, Z n , Z n are independent. By Proposition 2.2, the total variation distance between the two processes is bounded above by the expectation of the total variation with respect to X n−1 , X n−1 , Z n , Z n .
For n ≥ 2, suppose that Z n , Z n are independent. By Proposition 2.2, the total variation distance between the two processes is bounded above by the expectation of the total variation.