{'fraction_non_alphanumeric': 0.06588671325548684, 'fraction_numerical': 0.0186429703721896, 'mean_word_length': 3.870066325975862, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 38, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Empirical process theory for locally stationary processes

We provide a framework for empirical process theory of locally stationary processes using the functional dependence measure. Our results extend known results for stationary Markov chains and mixing sequences by another common possibility to measure dependence and allow for additional time dependence. Our main result is a functional central limit theorem for locally stationary processes. Moreover, maximal inequalities for expectations of sums are developed. We show the applicability of our theory in some examples, for instance we provide uniform convergence rates for nonparametric regression with locally stationary noise.


Introduction
Empirical process theory is a powerful tool to prove uniform convergence rates and weak convergence of composite functionals. The theory for independent variables is well-studied (cf. [19], [23], [44] or [43] for an overview) based on the original ideas of [13], [17], [18], [37] and [34] among others. For random variables with dependence structure, various approaches have been discussed. There exists a well-developed empirical process theory and large deviation results for Harris-recurrent Markov chains based on regenerative schemes (cf. [28], [41], [21] and [1] among others) or geometric ergodicity (cf. [27]). To quantify the speed of convergence in maximal inequalities, additional assumptions like β-recurrence (cf. [26]) have to be imposed. The theory covers a rich class of Markov chains, but for instance, can not discuss linear processes.
An empirical process theory for stationary processes under high-level assumptions on the moments of means was derived in [12] and further discussion papers. In the paradigm of weak dependence (which measures the size of covariances of Lipschitz functions of the random variables), [16] derived Bernstein-type inequalities. Focusing on the analysis of the empirical distribution function (EDF), much more techniques were discussed: For instance, [20,Theorem 4] provide uniform convergence of the EDF by using bounds for covariances of Hölder functions of the random variables. Another abstract concept was introduced by [4] via S-mixing (for stationary mixing), which imposes the existence of m-dependent approximations of the original observations. They then derive strong approximations and uniform central limit theorems for the EDF.
In [10] it is argued that β-mixing is the weakest mixing assumption that allows for a "complete" empirical process theory which incorporates maximal inequalities and uniform central limit theorems. There exist explicit upper bounds for β-mixing coefficients for Markov chains (cf. [25]) and for so-called V-geometric mixing coefficients (cf. [32]). For several stationary time series models like linear processes (cf. [35] for α-mixing), ARMA (cf. [33]), nonlinear AR (cf. [26]) and GARCH processes (cf. [22]) there also exist upper bounds on mixing coefficients. A common assumption in these results is that the observed process or, more often, the innovations of the corresponding process, have a continuous distribution. This is a crucial assumption to handle the relatively complicated mixing coefficients defined over a supremum over two different sigma-algebras. A relaxation of β-mixing coefficients was investigated by [11,Theorem 1] and is specifically designed for the analysis of the EDF. In opposite to sigma-algebras, these smaller coefficients are defined with conditional expectations of certain classes of functions and are easier to upper bound for a wide range of time series models.
During the last years, another measure for dependence, the so-called functional dependence measure, became popular (cf. [46]), which uses a Bernoulli shift representation (see (1.1) below) and decomposition into martingales and m-dependent sequences. It has been shown in various applications that the functional dependence measure allows, when combined with the rich theory of martingales, for sharp large deviation inequalities (cf. [49] or [51]). In [47] and [31], uniform central limit theorems for the EDF were derived for stationary and piecewise locally stationary processes.
Up to now, no general empirical process theory (allowing for general classes of functions) using the functional dependence measure is available. In this paper we will fill this gap and prove maximal inequalities and functional central limit theorems under functional dependence. Furthermore, we will draw connections and compare our results to already existing empirical process concepts for dependent data we mentioned above. While the empirical process theory for Markov chains and mixing cited above was developed for stationary processes, we will work in the framework of locally stationary processes and therefore automatically provide the first general empirical process theory in this setting ( [7] investigated spectral empirical processes for linear processes, [31] proved a functional central limit theorem for a localized empirical distribution function). Locally stationary processes allow for a smooth change of the distribution over time but can locally approximated by stationary processes. Therefore, they provide more flexible time series models (cf. [8] for an introduction).
The functional dependence measure uses a representation of the given process as a Bernoulli shift process and quantifies dependence with a L ν -norm. More precisely, we assume that X i = (X ij ) j=1,...,d , i = 1, ..., n, is a d-dimensional process of the form where G i = σ(ε i , ε i−1 , ...) is the sigma-algebra generated by ε i , i ∈ Z, a sequence of i.i.d. random variables in Rd (d,d ∈ N), and some measurable function J i,n : (Rd) N 0 → R, i = 1, ..., n, n ∈ N. For a real-valued random variable W and some ν > 0, we define W ν := E[|W | ν ] 1/ν . If ε * k is an independent copy of ε k , independent of ε i , i ∈ Z, we define G Graphically, δ X ν measures the impact of ε 0 in X k . Although representation (1.1) appears to be rather restrictive, it does cover a large variety of processes. In [5] it was motivated that the set of all processes of the form X i = J(ε i , ε i−1 , ...) should be equal to the set of all stationary and ergodic processes. We additionally allow J to vary with i and n to cover processes which change their stochastic behavior over time. This is exactly the form of the so-called locally stationary processes discussed in [8].
Since we are working in the time series context, many applications ask for functions f that not only depend on the actual observation of the process but on the whole (infinite) past Z i := (X i , X i−1 , X i−2 , ...). In the course of this paper, we aim to derive asymptotic properties of the empirical process Let H(ε, F, · ) denote the bracketing entropy, that is, the logarithm of the number of ε-brackets with respect to some distance · that is necessary to cover F (this is made precise at the end of this section). We will define a distance V n which guarantees weak convergence of (1.3) if the corresponding bracketing entropy integral The definition of the functional dependence measure for locally stationary processes is similar to its stationary version and is easy to calculate for many time series models. It does not rely on the stationarity assumption but on the representation of the process as a Bernoulli shift. Therefore, many well-known upper bounds for stationary time series given in [48], including recursively defined models and linear models, directly carry over to the locally stationary case (1.2). It seems reasonable to use it as a starting point to generalize empirical process theory for stationary processes to the more general setting of local stationarity. While the other two paradigms mentioned above should also allow such a generalization in principle, there are open questions: • The theory for Harris-recurrent Markov chain relies on stationarity and intrinsically needs some knowledge about the whole time series due to the assumption of null-recurrence. There exist generalizations of local stationary Markov chains (cf. for instance [42]), but the corresponding recurrence properties and examples of locally stationary time series are not worked out yet. Furthermore, it is not directly clear how to deal with processes Z i which incorporate the infinite past of X i , and linear processes can not be discussed easily.
• Absolutely regular β-mixing is shown in most examples by assuming some continuity in the distribution or in the corresponding innovations. Especially for linear processes, the bounds are quite hard to obtain and seem not to be optimal. Moreover, there exist no "invariance rules" which would directly allow to transfer the mixing properties of X i to f (Z i , i n ) which incorporates infinitely many lags of X i .
• Many of the more elaborated dependence concepts developed in e.g. [11], [6], [20], [4] are restricted (at least in their original formulation) to the discussion of the EDF or connected one-dimensional indexed function classes.
Contrary to β-mixing, the functional dependence measure can easily deal with f (Z i , i n ) by using Hölder-type assumptions on f . Furthermore, it easily can be calculated in many situations and is not restricted to noncontinuous distributions of X i . Also, linear processes X i are covered.
However, there are also some peculiarities using the functional dependence measure (1.2). While for Harris-recurrent Markov chains and β-mixing, the empirical process theory is independent of the function class considered, the situation for the functional dependence measure is more complicated. In order to quantify the dependence of f (Z i , i n ) by δ X ν , we have to impose smoothness conditions on f in direction of its first argument. The distance V n therefore will not only change with the dependence structure of X, but also has to be "compatible" with the function class F. The smoothness condition on f also poses a challenging issue when considering chaining procedures where rare events are excluded by (non-smooth) indicator functions.
Our main contributions in this paper are the following: • We derive maximal inequalities for G n (f ) for classes of functions F, • a chaining device which preserves smoothness during the chaining procedure and • conditions to ensure asymptotic tightness and functional convergence of G n (f ), f ∈ F.
The paper is organized as follows. In Section 2, we present our main result Theorem 2.3, the functional central limit theorem under minimal moment conditions. As a special case, we derive a version for stationary processes. We give a discussion on the distance V n and compare our result with the empirical process theory for β-mixing. Some Assumptions are postponed to Section 3, where a new multivariate central limit theorem for locally stationary processes is presented. In Section 4, we provide new maximal inequalities for G n (f ) for both finite and infinite F. In Section 5, we apply our theory to prove uniform convergence rates and weak convergences of several estimators. The aim of the last section is to highlight the wide range of applicability of our theory and to provide the typical conditions which have to be imposed as well as some discussion. In Section 6, a conclusion is drawn. We illustrate the main steps of the proofs in the Appendix of the article but postpone all detailed proofs to the Supplementary Material.
We now introduce some basic notation. For a, b ∈ R, let a ∧ b := min{a, b}, a ∨ b := max{a, b}. For k ∈ N, which naturally appears in large deviation inequalities. For a given finite class F, let |F| denote its cardinality. We use the abbreviation if no confusion arises. For some distance · , let N(ε, F, · ) denote the bracketing numbers, that is, the smallest number of ε-brackets [l j , u j ] := {f ∈ F : denote the bracketing entropy. For ν ≥ 1, let

A new functional central limit theorem
Roughly speaking, a process X i , i = 1, ..., n is called locally stationary if for each u ∈ [0, 1], there exists a stationary processX i (u), i = 1, ..., n such that X i ≈X i (u) if |u − i n | is small (cf. [8]). Typical estimators are of the form where K is a kernel function and h = h n > 0 is a bandwidth. Clearly, such a localization changes the convergence rate. To cover these cases, we suppose that any f ∈ F has a representation wheref is independent of n and D f,n (u) is independent of z. We put The function classF is considered to consist of Hölder-continuous functions in direction of z. For s ∈ (0, 1], a sequence z = (z i ) i∈N 0 of elements of R d (equipped with the maximum norm | · | ∞ ) and an absolutely summable sequence χ = (χ i ) i∈N 0 of nonnegative real numbers, we set and |z| χ := |z| χ,1 .
Definition 2.1.F is called a (L F , s, R, C)-class if L F = (L F ,i ) i∈N 0 is a sequence of nonnegative real numbers, s ∈ (0, 1] and R : The basic assumption for our main result is the following compatibility condition on F. While (2.3) summarizes moment assumptions on X ij which are balanced by p, the sequence ∆(k) reflects the intrinsic dependence of f (Z i , i n ) and D n measures the influence of the factor D f,n (u) to the convergence rate of G n (f ).
Based on Assumption 2.2, we define for f ∈ F V n (f ) := f 2,n + ∞ k=1 min{ f 2,n , D n ∆(k)}. (2.4) Clearly, V n satisfies a triangle inequality. Therefore, V n (f − g) is a distance between f, g ∈ F. We are now able to state our main result. The weak convergence takes place in the normed space Then it holds in ℓ ∞ (F) that where (G(f )) f ∈F is a centered Gaussian process with covariances The proof of Theorem 2.3 consists of two ingredients, convergence of the finite-dimensional distributions (cf. Theorem 3.4) and asymptotic tightness (cf. Corollary 4.5). The more challenging part is asymptotic tightness; its proof only relies on Assumption 2.2 and consists of a new maximal inequality presented in Theorem 4.1 which may be of independent interest. To ensure convergence of the finite-dimensional distributions, we have to formalize local stationarity (Assumption 3.1) and pose conditions in time direction onf (z, ·) (cf. Assumption 3.2) and D f,n (·) (cf. Assumption 3.3) which is done in Section 3. In particular, it is needed that D f,n (u) is properly normalized.
Let us note that in the case that X i is stationary,f (z, u) =f (z) and D f,n (u) = 1, Assumptions 3.1, 3.2 and 3.3 are directly fulfilled. That is, in the stationary case, Assumption 2.2 is sufficient for Theorem 2.3. We formulate this finding as a simple corollary. Let where X i = J(G i ), i = 1, ..., n, is a stationary process and h are functions from where (G(h)) h∈H is a centered Gaussian process with covariances 2.1 Form of V n and discussion on ∆(k)

Form of V n
Suppose that D n ∈ (0, ∞) is independent of n ∈ N. Based on decay rates of ∆(k), simpler forms of V n can be derived and are given in Table 1. These results are elementary and are proved in Lemmas 7.11,7.12 in the Supplementary Material. Table 1: Equivalent expressions of V n and the corresponding entropy integral under the condition that D n ∈ (0, ∞) is independent of n. We omitted the lower and upper bound constants which are only depending on c, ρ, α and D n . Furthermore,σ =σ(σ) fulfillsσ → 0 for σ → 0.
If f (Z i , i n ), i = 1, ..., n, are independent, we can choose ∆(k) = 0 for k ≥ 1 and thus V n (f ) is proportional to f 2,n . We therefore exactly recover the case of independent variables with our theory.

Discussion on ∆(k)
which is a convolution of the uniform Hölder constants L F ,j of f ∈ F and the dependence measure δ X 2sp p−1 (k) of X. Therefore, the specific form of f ∈ F has an impact on the dependence structure which is then introduced in V n . This is contrary to other typical chaining approaches for Harris-recurrent Markov chains or β-mixing sequences where the dependence structure of X i simply transfers to functions f (X i ) without further conditions. Furthermore, contrary to other chaining approaches, we have to ask for the existence of moments of X i in Assumption 2.2 even though G n (f ) only involves f (X i ). This is due to the linear nature of the functional dependence measure (1.2). If f is Lipschitz continuous with respect to its first argument (s = 1 in Assumption 2.2), we have to impose sup i,j X ij 2 < ∞. However, these moment assumptions can be relaxed at the cost of larger ∆(k) as follows. Let us consider the special case that f (Z i , i n ) only depends on X i , i n , that is, f (z, u) = f (z 0 , u). If f is bounded and Lipschitz continuous with respect to its first argument with Lipschitz constant L, for any s ∈ (0, 1], Thus, ∆(k) can be chosen proportional to δ X 2s (k) s . This means that we can reduce the moment assumption to sup i,j X ij 2s < ∞ at the cost of having a larger norm V n .

Comparison to empirical process theory with β-mixing
In this section, we compare our functional central limit theorem for stationary processes from Corollary 2.4 under functional dependence with similar results obtained under β-mixing. Unfortunately, we were not able to find a general setting under which the functional dependence measure δ X 2 can be compared with the β-mixing coefficients β X of X i , i = 1, ..., n. However, in some special cases, both quantities can be upper bounded.

Upper bounds for dependence coefficients of linear processes
Consider the linear process with an absolutely summable sequence a k , k ∈ N 0 , and i.i.d. ε k , k ∈ Z, with Eε 1 = 0. Then it is immediate that δ X 2 (k) ≤ 2|a k | · ε 1 2 . From [35] (cf. also [15], Section 2.3.1), we have the following result. If for some ν ≥ 1, ε 1 ν < ∞, ε 1 has a Lipschitz-continuous Lebesgue-density and the process X i is invertible, then for some constant ζ > 0, where A m,s := ∞ k=m |a k | s and L(u) = u(1 ∨ | log(u)|). If a k = O(k −α ) for some α > 1, Note that even for this specific example, the calculation of the functional dependence measure is much easier and possible with much less assumptions. Moreover, bounds for β X (k) is typically larger than δ X 2 (k). The reason being the simple structure of δ X 2 compared to the much more involved formulation of dependence through sigma-algebras in the β-mixing coefficients. For recursively defined processes with a finite number of lags, δ X 2 are typically upper bounded by geometric decaying coefficients (cf. [48], [8]); the same holds true for β X (k) under additional continuity assumptions (cf. [15], Section 2.4. or [27], [25] among others).

Entropy integral
In [14] (cf. also [10]), it was shown that if X i , i = 1, ..., n, is stationary and β-mixing with coefficients β(k), k ∈ N 0 , then implies weak convergence of (G n (h)) h∈H in ℓ ∞ (H). Here, the · 2,β -norm is defined as follows. If β −1 denotes the inverse cadlag of the decreasing function t → β(⌊t⌋) and Q h the inverse cadlag of the tail function t → P(h(X 1 ) > t), then Condition (2.8) was later relaxed in [40,Theorem 8.3]. It could be shown that if F consists of indicator functions of specific classes of sets (in particular, F corresponds to the empirical distribution function), weak convergence can be obtained under less restrictive conditions than (2.8). Since our theory does not directly allow us to analyze indicator functions because F has to be a (L F , s, R, C)-class, we do not discuss their generalization here in detail.
In the special cases of polynomial and geometric decay, simple upper bounds for h 2,β are available (cf. [10]).
Generally speaking, (2.8) asks for 2b b−1 moments of the process f (X i ) to exist while our condition in (2.6) only asks for 2 moments of f (X i ) but allows for smaller function classes through the additional factors given in the entropy integral (cf. Table 1). In specific examples (cf. (2.7)) it may occur that the entropy integral (2.6) is finite while (2.8) is infinite due to missing summability of β X (k).
To give a precise comparison, consider the situation of linear processes from (2.7). If ν > 2α + 1, we can choose b = α − 3 2 . Then, the two entropy integrals from Corollary 2.4 (left) and (2.8) read Here, the entropy integral for mixing only exists if α > 5 2 . The difference in the behavior is due to different bounds used for the variance of G n (f ).

Integration into other empirical process results for the empirical distribution function of dependent data
While our approach does allow for a general empirical process theory for Hölder continuous function classes, some more general dependence concepts already have been introduced (only) for the discussion of the empirical distribution function (EDF) based on the one-dimensional class We mention [6], [9], [4] and [20]. The conditions therein cover the case and its summed up version, D ν (k) := ∞ j=k+1 δ X ν (j). [6] introduces so-called ν-approximation coefficients a k , k ∈ N, which can be viewed as another formulation of functional dependence. However, their final result [6,Theorem 5] for the convergence of the EDF is stated with summability conditions both on a k and absolutely regular mixing coefficients, we therefore do not discuss it in detail here. On the other hand, [9, Theorem 2.1] in combination with [11,Section 6.1] show convergence of the EDF if for some ν > 1 and γ > 0, This is done by introducing simplified β-mixing coefficients which can then be upper bounded by D ν (k). By using independent approximations of the original process, [4, Theorem 1, Corollary 1] obtain convergence of the EDF if for some ν ≥ 1 and A > 4, (D ν (k)k A ) ν ≤ k −A , or equivalently, [20] discusses convergence of the EDF under a general growth condition imposed on the moments of n i=1 {h(X i ) − Eh(X i )} where h ∈ H α , the set of all Hölder-continuous functions with exponent α ∈ (0, 1]. Their condition is fulfilled if

A general central limit theorem for locally stationary processes
In this section, we introduce the remaining assumptions needed in Theorem 2.3 which pose regularity conditions on the process X i and the function class F in time direction. They are used to derive a multivariate central limit theorem for (G n (f 1 ), ..., G n (f k )) under minimal moment conditions in Theorem 3.4. Comparable results in different and more specific contexts were shown in [8] or [42].
We first formalize the property of X i to be locally stationary (cf. [8]). We ask that for each u ∈ [0, 1] there exists a stationary processX i (u), i = 1, ..., n, such that X i ≈X i (u) if |u − i n | is small. Recall R(·), s, p from Assumption 2.2.
One of the two following cases hold.
has bounded variation for all j 1 , j 2 ∈ N 0 and the following limit exists: The following limit exists for all f, g ∈ F: Assumption 3.3 looks rather technical. The first part including (3.1) guarantees the right normalization of D f,n (·). The second part ensures the convergence of the asymptotic variances Var(G n (f )) and covariances Cov(G n (f ), G n (g)).
We obtain the following central limit theorem.
Theorem 3.4 generalizes the one-dimensional central limit theorem from [8]. We now comment on the assumptions.
Remark 3.5. Assumptions 3.1, 3.2 and 3.3 allow for very general structures of f ∈ F. However, in many special cases, a subset of them is automatically fulfilled: • If X i is stationary, then Assumption 2.2 already implies Assumption 3.1.
• Iff (z, u) =f (z) does not depend on u, Assumption 3.2 is fulfilled.

Maximal inequalities and asymptotic tightness under functional dependence
In this section, we provide the necessary ingredients for the proof of asymptotic tightness of G n (f ). We derive a new maximal inequality for finite F under functional dependence in Theorem 4.1. We then generalize this bound to arbitrary F using chaining techniques in Section 4.2.

Maximal inequalities
We first derive a maximal inequality which is a main ingredient for chaining devices but also is of independent interest. To state the result, let Theorem 4.1. Suppose that F satisfies |F| < ∞ and Assumption 2.2. Then there exists some universal constant c > 0 such that the following holds: and Clearly, the second bound (4.3) is a corollary of (4.2) which balances the two terms which involve q. Values of q * (·) for the two prominent cases that ∆(·) is polynomial or exponential decaying can be found in Table 2. The proof of Theorem 4.1 relies on a decomposition of G n (f ) in i.i.d. parts and a residual term of martingale structure. Similar decompositions are also the core of empirical process results for Harris-recurrent Markov chains (cf. [29]) and mixing sequences (cf. [10]). Table 2: Equivalent expressions of q * (·) and r(·) taken from Lemma 7.10 in Section 7.9. We omitted the lower and upper bound constants which are only depending on C, ρ, α.

∆(j)
In the next subsections, we will prove asymptotic tightness for G n (f ) under the condition that D ∞ n , D n do not depend on n. However, uniform convergence rates of G n (f ) for finite F (growing with n) can be obtained without these conditions but with additional moment assumptions, which is done in the following Corollary 4.3. To incorporate the additional moment assumptions, we use a slightly stronger assumption than Assumption 2.2.
Note that Assumption 2.2 is obtained by taking ν = 2. For δ > 0, let cf. Table 2 for values of r(·) in special cases.
Then, max The first condition in (4.5) guarantees that G n (f ) is properly normalized. The second and third condition are needed to prove that the "rare events", where |f (Z i , i n )| exceeds some threshold M n ∈ (0, ∞), are of the same order as √ H. For this, we may need more than two moments of f (Z i , i n ), that is, ν > 2, depending on √ H and the behavior of D ∞ n .
Corollary 4.3 can be used to prove (optimal) convergence rates for kernel density and regression estimators as well as maximum likelihood estimators under dependence. We give an example in Section 5.

Asymptotic tightness
In this section, we extend the maximal inequality from Theorem 4.1 to arbitrary (infinite) classes F. Since Assumption 2.2 forces f ∈ F to be Hölder-continuous with respect to its first argument z, classical chaining approaches which use indicator functions do not apply here. We provide a new chaining technique which preserves continuity in Section 7.2.
For n ∈ N, δ > 0 and k ∈ N define H(k) = 1 ∨ log(k) and Here, m(n, δ, k) represents the threshold for rare events in the chaining procedure. We have the following result.
Theorem 4.4. Let F satisfy Assumption 2.2 and let F be some envelope function of F, that is, for each f ∈ F it holds that |f | ≤ F . Let σ > 0 and assume that sup f ∈F V n (f ) ≤ σ. Then there exists some universal constantc > 0 such that As a corollary, we obtain asymptotic equicontinuity of G n (f ). Here, we use Assumptions 3.1 and 3.2 only to discuss the remainder term in Theorem 4.4 without imposing the existence of additional moments.
Corollary 4.5. Let F satisfy Assumption 2.2, 3.1 and 3.2. Suppose that Furthermore, assume that D n , D ∞ n ∈ (0, ∞) are independent of n, and Then, the process G n (f ) is equicontinuous with respect to V n , that is, for every η > 0,

Applications
In this section, we provide some applications of the main results (Corollary 4.3 and Corollary 2.3). We will focus on locally stationary processes and therefore use localization in our functionals, but the results also hold for stationary processes, accordingly.
Let K : R → R be some bounded kernel function which is Lipschitz continuous with Lipschitz In the first example we consider the nonparametric kernel estimator in the context of nonparametric regression with fixed design and locally stationary noise. We show that under conditions on the bandwidth h, which are common in the presence of dependence (cf. [24] or [45]), we obtain the optimal uniform convergence rate log(n) nh . Write a n b n for sequences a n , b n if there exists some constant c > 0 such that a n ≥ cb n for all n ∈ N.

Suppose that either
• δ X 2 (j) ≤ κj −α with some κ > 0, α > 1, and h ( log(n) First note that due to Lipschitz continuity of K with Lipschitz constant L K , we have For the grid V n = {in −3 , i = 1, ..., n 3 }, which discretizes [0, 1] up to distances n −3 , we obtain by The conditions of Corollary 4.3 are easily verified: It holds that The conditions on h emerge from the last condition in (4.5) and using the bounds for r(·) from Table 2.
For the following two examples we suppose that the underlying process X i is locally stationary in the sense of Assumption 3.1. Similar assumptions are posed in [8] and are fulfilled for a large variety of locally stationary processes.
In the same spirit as in Example 5.1, it is possible to derive uniform rates of convergence for Mestimators of parameters θ in models of locally stationary processes. Furthermore, weak Bahadur representations can be obtained. The following results apply for instance to maximum likelihood estimation of parameters in tvARMA or tvGARCH processes. The main tool is to prove uniform convergence of the corresponding objective functions and its derivatives. Since the rest of the proof is standard, the details are postponed to the Supplementary Material, Section 7.8. Let ∇ j θ denote the j-th derivative with respect to θ. To apply empirical process theory, we ask for the objective functions to be (L F , 1, R, C)-classes in (A1) and Lipschitz with respect to θ in (A2). For each θ ∈ Θ, let ℓ θ : R k → R be some measurable function which is twice continuously differentiable. Let Z i = (X i , ..., X i−k+1 ), and define for v ∈ [0, 1], Furthermore, suppose that either which for a ∈ (−1, 1) is a ((1, a), 1, |x 0 | + |x 1 |, (0, 1))-class.
• With more smoothness assumptions on ∇ θ ℓ or using a local linear estimation method forθ n,h , the bias term B h can be shown to be of smaller order, for instance O(h 2 ) (cf. [8]).
• The theory derived in this paper can also be used to prove asymptotic properties of Mestimators based on objective functions ℓ θ which are only almost everywhere differentiable in the Lebesgue sense by following the theory of chapter 5 in [43]. This is of utmost interest for ℓ θ that have additional analytic properties, such as convexity. Since these properties are also needed in the proofs, we will not discuss this in detail.
We give an easy application of the functional central limit theorem from Theorem 2.3 by inspecting a local stationary version of Example 19.25 in [43].

Conclusion
In this paper, we have developed a new empirical process theory for locally stationary processes with the functional dependence measure. We have proven a functional central limit theorem and maximal inequalities. A general empirical process theory for locally stationary processes is a key step to derive asymptotic and nonasymptotic results for M-estimates or testing based on L 2 -or L ∞statistics. We have given an example in nonparametric estimation where our theory is applicable.
Due to the possibility to analyze the size of the function class and the stochastic properties of the underlying process separately, we conjecture that our theory also permits an extension of various results from i.i.d. to dependent data, such as empirical risk minimization.
From a technical point of view, the linear and moment-based nature of the functional dependence measure has forced us to modify several approaches from empirical process theory for i.i.d. or mixing variables. A main issue was given by the fact that the dependence measure only transfers decay rates for continuous functions. We therefore have provided a new chaining technique which preserves continuity of the arguments of the empirical process.
In principle, a similar empirical process theory for locally stationary processes can be established under mixing conditions such as absolute regularity. This would be a generalization of the results found in [38] and [10]. As we have seen in Section 2.2, such a theory would pose additional moment conditions on f (Z i , i n ). Contrary to that, our framework only requires second moments of f (Z i , i n ), but the entropy integral is enlarged by some factor which increases with stronger dependence. Moreover, in nearly all models the derivation of a bound for mixing coefficients needs continuity of the innovation process which may not be suitable in several examples. Therefore, we consider our theory as a valuable addition to this existing theory even in the stationary case.
One could also think of an extension of our empirical process theory to functions f (z, u) which are noncontinuous with respect to z. This can in principle be done by using a martingale decomposition and assuming continuity of instead. However, one typically can only expect continuity of this functional if either already z → f (z, u) was continuous or ε i has a continuous density. In the latter case, the sequence might also be β-mixing, and a more detailed discussion about advantages of our formulation is necessary.

Appendix
In the appendix, we present the basic ideas used to prove the maximal inequalities of Section 4. We first consider the finite version in Section 7.1 and then present a chaining approach which preserves continuity in Section 7.2.

Proof idea: A maximal inequality for finite F , Theorem 4.1
We provide an approach to obtain maximal inequalities for sums of random variables W i (f ), i = 1, ..., n, indexed by f ∈ F, by using a decomposition into independent random variables. An approach with similar intentions is presented in [10] (Section 4.3 therein) for absolutely regular sequences and in [29] for Harris-recurrent Markov chains. For convenience, we abbreviate To approximate W i (f ) by independent variables, we use a technique from [49] which was refined in [51]. This decomposition is much more involved then the ones for Harris-recurrent Markov chains or mixing sequences since no direct coupling method is available. Define Let q ∈ {1, ..., n} be arbitrary. Put L := ⌊ log(q) log(2) ⌋ and τ l := 2 l (l = 0, ..., L − 1), τ L := q. Then we have (in the case q = 1, the sum in the middle does not appear) and thus We write While the first term in (7.1) can be made small by assumptions on the dependence of W i (f ) and by the use of a large deviation inequality for martingales in Banach spaces from [36], the second and third term allow the application of Rosenthal-type bounds due to the independency of the summands T i,l (f ) and W i,1 (f ), respectively. Since the first term in (7.1) allows for a stronger bound in terms of n than it is the case for mixing, we can obtain a theory which only needs second We now summarize the proof of Theorem 4.1. The detailed proof is found in the Supplementary material. Denote The remaining terms in (7.1) are discussed similarly or are special cases. We first have By Theorem 4.1 in [36] there exists an absolute constant c 1 > 0 such that for s > 1, By using (7.2) and standard techniques for the functional dependence measure, we conclude that Summarizing the results, we obtain Regarding A 2 , we have Using the fact that martingale sequences are uncorrelated, (7.3) and the simple bound |W i (f )| ≤ M , one can show that A maximal inequality for independent random variables based on Bernstein's inequality yields that there exists some universal constant c 2 > 0 such that By monotonicity of the first term with respect to f 2,n and we obtain with some universal constant c 3 > 0, which together with (7.5) provides the result of Theorem 4.1.

An elementary chaining approach which preserves continuity
In this section, we provide a chaining approach which preserves continuity of the functions inside the empirical process. Typical chaining approaches work with indicator functions which is not suitable for application of Theorem 4.1. We replace the indicator functions by suitably chosen truncations. For m > 0, define ϕ ∧ m : R → R and the corresponding "peaky" residual ϕ ∨ m : In the following, assume that for each j ∈ N 0 there exists a decomposition F = Assume furthermore that there exists a sequence (∆ j f ) j∈N such that for all j ∈ N 0 , sup f,g∈F jk |f − g| ≤ ∆ j f . Finally, let (m j ) j∈N 0 be a decreasing sequence which will serve as a truncation sequence.
For j ∈ N 0 , we use the decomposition we can write where To bound R(j), we use (i) of the following elementary Lemma 7.1 which is proved in Section 7.6 included in the Supplementary Material.
Lemma 7.1. Let y, x, x 1 , x 2 , x 3 and m, m ′ > 0 be real numbers. Then the following assertions hold: Because the partitions are nested, we have |π j+1 f − π j f | ≤ ∆ j f . By Lemma 7.1 and (7.6), we have Let τ ∈ N. We then have with iterated application of (7.7) and linearity of f → W i (f ), G n (R(j)), (7.9) which in combination with (7.8) can now be used for chaining. The following lemma provides the necessary balancing between the truncated versions of G n (f ) and the rare events excluded. Recall that H(k) = 1 ∨ log(k) as in (1.4).  , which shows (7.10). Since for all f ∈ F with V n (f ) ≤ δ, it holds that In the case f 2,n < D n ∆(1), the fact that ∆(·) is decreasing implies that a * = max{j ∈ N : = f 2,n (a * + 1) + D n β(a * ) ≥ f 2,n a * + β(a * ). (7.14) Summarizing the results (7.13) and (7.14), we have V n (f ) ≥ f 2,n (a * ∨ 1) + D n β(a * ∨ 1).

Proof idea: A maximal inequality for infinite F , Theorem 4.4
The details of the proof are given in Section 7.5 in the Supplementary material. In the following, we abbreviate H(δ) = H(δ, F, V n ) and N(δ) = N(δ, F, V n ). Choose δ 0 = σ and δ j = 2 −j δ 0 .
For each j ∈ N 0 , we choose a covering by brackets F jk := [l jk , u jk ] ∩ F, k = 1, ..., N j := N(δ j ) such that V n (u jk − l jk ) ≤ δ j and sup f,g∈F jk |f − g| ≤ u jk − l jk =: ∆ jk . We may assume w.l.o.g. that l jk , u jk , ∆ jk ∈ F and that (F jk ) k are nested.

Supplementary Material
This material contains some details of the proofs in the paper as well as the proofs of the examples.

Proofs of Section 4.1
Lemma 7.3. Let Assumption 4.2 hold for some ν ≥ 2. Then, Proof of Lemma 7.3. We have for each f ∈ F and ν ≥ 2 that This shows the first assertion. Due to the second assertion follows similarly. The last assertion follows from Proof of Theorem 4.1. Denote the three terms on the right hand side of (7.1) by A 1 , A 2 , A 3 . We now discuss the three terms separately. First, we have For fixed j, the sequence . For a vector x = (x f ) f ∈F and s ≥ 1, write |x| s := ( f ∈F |x f | s ) 1/s . By Theorem 4.1 in [36] there exists an absolute constant c 1 > 0 such that for s > 1, therefore both terms in (7.18) are of the same order and it is enough to bound the second term in (7.18). We have Note that where By Jensen's inequality, Lemma 7.3 and the fact that (W i (f ) * * (i−j) , W i (f ) * * (i−j+1) ) has the same distribution as (W i (f ), W i (f ) * (i−j) ), Inserting this bound into (7.18), we obtain We conclude with s := 2 ∨ log |F| that We now discuss EA 2 . If M Q , σ Q > 0 are constants and Q i (f ), i = 1, ..., m mean-zero independent variables (depending on Note that (W k,j −W k,j−1 ) k is a martingale difference sequence and W k,τ l −W k,τ l−1 = τ l j=τ l−1 +1 (W k,j − W k,j−1 ). Furthermore, we have We conclude with the elementary inequality min{a and a similar assertion for the second term (i odd) in A 2 . With (7.24), we conclude that min{x, D n ∆(j)} (7.28) and the second to last equality holds since x →V n (x) is increasing. Inserting (7.26) and (7.27) into (7.25), we conclude that with some universal c 3 > 0, , we obtain from (7.23) again If we insert the bounds (7.22), (7.29) and (7.30) into (7.1), we obtain the result (4.2).
Especially in the case ω(k) = 1, Proof of Lemma 7.4. It holds that LetF = sup f ∈Ff , and F (z, u) = D ∞ n (u) ·F (z, u). Then F is an envelope function of F. We furthermore have Inserting the bound into (7.33) and using r(γa) ≥ γr(a) for γ ≥ 1, a > 0 (this is similarly proven as for Lemma 7.5), we obtain Using the rough bound f ν,n ≤ F ν,n and r(a) ≤ a for a > 0 from Lemma 7.5, we obtain By assumption, sup n∈N C n < ∞. By Theorem 4.1, (7.34) and (7.35), Since sup n∈N C n < ∞ and σ is independent of n, the assertion follows for Q → ∞.
If l jk , u jk do not belong to F, we can simply define new brackets bỹ Thus, we can addl jk ,ũ jk to F without changing the bracketing numbers N(ε, F, · ) and the validity of Assumption 4.2.
We now construct inductively a new nested sequence of partitions (F jk ) k of F from (F pre jk ) k in the following way: For each fixed j ∈ N 0 , put as the intersections of all previous partitions and the j-th partition. Then |{F jk : k}| ≤ N j := N(δ 0 ) · ... · N(δ j ). By monotonicity of V n , we have In each F jk , fix some f jk ∈ F, and define π j f := f j,ψ j f where ψ j f := min{i ∈ {1, ..., N j } : f ∈ F ji }.
(iii) We will show that for all z, z ′ ∈ R N it holds that from which the assertion follows. For real numbers a i , b i , we have This implies | max{a, y} − max{a, y ′ }| ≤ |y − y ′ | and therefore For the second inequality in (7.51), note that We therefore have A similar result is obtained for f (z) ≥ m, f (z ′ ) ≤ m, which proves (7.51).
Let a > 0. We show that r = 2r( a 2 ) fulfills q * (r)r ≤ a. By definition of r(a), we obtain r(a) ≥ r = 2r( a 2 ) which gives the result. Since β norm is decreasing, q * is decreasing. We conclude that q * (r)r = 2 · q * (2r( a 2 ))r( The second inequality r(a) ≤ a follows from the fact that q * (r)r is increasing and q * (a)a ≥ a.

Proofs of Section 3
Proof of Theorem 3.4. = (a 1 , ..., a m ) ′ ∈ R m \{0}. We use the decomposition Then, since P i−j W i (f k ), i = 1, ..., n is a martingale difference sequence and by Lemma 7.8(i), Then we have which gives lim n→∞ S • n (J) k − S n (J) k 2 = 0. (7.53) Then we have For each j, k, it holds that By Lemma 7.8, we have sup i f (Z i+j , i+j n ) 2 2 < ∞. Since 1 √ n D f k ,n (·) has bounded variation uniformly in n, We therefore obtain is a martingale difference sequence with respect to G i−1 , and We can therefore apply a central limit theorem for martingale difference sequences to a ′S• n (J) = The Lindeberg condition: Let ς > 0. Iterated application of Lemma 7.6(i) yields that there are constants c 1 , c 2 > 0 only depending on m, J such that For each l, j, k, we have where we have put By Lemma 7.8(ii),W i (u) satisfies the assumptions (7.59) of Lemma 7.7. By assumption, c n → ∞.
With a n (u) := D f k ,n (u) 2 , we obtain from Lemma 7.7 that (7.55) converges to 0, which shows that the Lindeberg condition is satisfied.
and thus Here, for f, g ∈ F, we have that E[P 0f (Z j 1 (u), u) · P 0ḡ (Z j 2 (u), u)] can be written as which shows that the condition stated in the assumption guarantees the bounded variation of u → E[P 0f (Z j 1 (u), u) · P 0ḡ (Z j 2 (u), u)].
Case K = 2: If h n → 0, then we obtain similarly kl (J)a.
(i) For x, y ∈ R, it holds that (ii) For random variables W,W , it holds that Proof of Lemma 7.6. (i) It holds that . Inserting this inequality into (7.58), we obtain the assertion.
The following lemma generalizes some results from [8] using similar techniques as therein.
Let a n : [0, 1] → R be some sequence of functions with lim sup n→∞ 1 n n i=1 |a n ( i n )| < ∞.
(i) Let q = 2. Let c n be some sequence with c n → ∞. Then (ii) Let q = 1. Suppose that there exists h n > 0, v ∈ [0, 1] such that for all u ∈ [0, 1], |v − u| > h n implies a n (u) = 0. Put A n = sup i=1,...,n |a n ( i n )| and suppose that sup n∈N (h n · A n ) < ∞, A n n → 0, a n (·) A n has bounded variation uniformly in n.
(ii) Since (7.59) also holds forW 0 (u) replaced byW 0 (u)−EW 0 (u), we may assume in the following that w.l.o.g. that EW 0 (u) = 0. By (7.61) applied to w i = a( i n )W i ( i n ), we obtain We furthermore have Fix j ∈ {1, ..., 2 J }. Put u j := j−1 2 J and, for a real-valued positive x, define [x] := max{k ∈ N : k > x}. By stationarity, the following equality holds in distribution: 1 |I j,J,n | i∈I j,J,n a n ( i n . By partial summation and since an(·) An has bounded variation B a uniformly in n, We conclude from (7.67) that Combination of (7.64), (7.65), (7.66) and (7.68) and applying lim sup n→∞ and afterwards lim sup J→∞ , we obtain If u → EW 0 (u) has bounded variation, we have with some intermediate value Since an(·) An has bounded variation uniformly in n, (ii) The second assertion follows from Lemma 7.3.
(ii) Geometric decay ∆(j) = κρ j (where ρ ∈ (0, 1)): Then there exist some constants c Proof of Lemma 7.11. The assertions follow from Lemma 7.9(ii) by taking κ 2 = κD n . The maximum in the lower bounds is obtained due to the additional summand f 2,n in V n (f ).
The following lemma formulates the entropy integral in terms of the well-known bracketing numbers in terms of the · 2,n -norm in the case that sup n∈N D n < ∞. For this, we use the upper bounds of V n given in Lemma 7.11.