{'fraction_non_alphanumeric': 0.06083164478900537, 'fraction_numerical': 0.019777993128358734, 'mean_word_length': 4.013913427561837, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Adaptive Energy Saving Approximation for Random Stationary Processes

We consider a stationary process (with either discrete or continuous time) and find an adaptive approximating stationary process combining approximation quality and supplementary good properties that can be interpreted as additional smoothness or small expense of energy. The problem is solved in terms of the spectral characteristics of the approximated process by using classical analytic methods from prediction theory.


Main objects and problem setting
Consider a random process B(t) with continuous (t ∈ R) or discrete (t ∈ Z) time. We try to approximate B with another process X that, being close to B, would have, appropriately understood, better sample path properties. For example, one may imagine that a sample path of B is a model of trajectory for some chaotically moving target while the sample path of X is a pursuit trajectory built upon observations of B. In the most interesting cases (when the time is continuous), the trajectories of B are non-differentiable, while the trajectories of X are required to be smooth.
In this article we assume that B and X are wide-sense stationary process. The additional requirements on X are stated in terms of small average expense of generalized energy, the latter notion being formalized below.
As usual, we assume that K B (·) is continuous.
We will look for an approximating process (X(t)) t∈R such that the pair (B, X) would be jointly wide-sense stationary, the processes be close to each other but X spends a small amount of energy (a notion to be specified soon) in its approximation efforts.
We call the instant energy of X at time t ∈ R an expression ℓ m X (m) (t) 2 , where X (m) stands for the m-th derivative of X and ℓ m are some fixed complex coefficients. The most natural type of energy is the kinetic energy which is just α 2 |X (1) (t)| 2 with some α > 0. The natural goal for us would be the minimization of the functional combining approximation and energy properties with averaging in time. If, additionally, the process X(t) − B(t) and all derivatives X (m) (t) are stationary processes in the strict sense, in many situations the ergodic theorem applies and the limit above is equal to E |X(0) − B(0)| 2 + E E[X](0). In the wide-sense theory, we simplify our task to solving the problem E |X(0) − B(0)| 2 + E E[X](0) ց min (1) and setting aside ergodicity issues. From the point of view of control theory, the term E E[X](0) may be considered as a sort of penalty imposing certain smoothness on X.
Notice that, once the problems of the form (1) are solved, one can easily separate the two terms in (1), solving (by Lagrange multipliers method) the somewhat more natural problems: a) Find a process X with minimal expense of energy and reaching prescribed closeness to B, for any given δ > 0, b) Find a process X reaching the best possible closeness to B using given amount of energy, for any given E > 0.
We will consider the problem (1) either in the simpler non-adaptive setting, i.e. without any further restrictions on X, or in the adaptive setting by requiring additionally where span{A|H} denotes the closed linear span of a subset A ⊂ H in a Hilbert space H. In other words, we only allow approximations based on the current and past values of B. The non-adaptive setting was considered in [5,6]. In the present paper, after briefly recalling the corresponding results, we concentrate on the much more interesting and difficult adaptive setting.
Our approach in solving (1) is based on spectral representations of stationary processes. We will now briefly recall some facts from this theory; see, e.g., [1].
According to the Bochner-Khinchine theorem, the covariance function K B (·) admits a spectral representation where µ is a finite measure called the spectral measure of B. Moreover, the process B itself admits a spectral representation where W(du) is a complex centered random measure with uncorrelated values on R controlled by the spectral measure µ, i.e. µ(A) = E |W(A)| 2 for any Borel set A ⊂ R.
Without loss of generality we may restrict our optimization to the class of approximating process having the form where g(·) ∈ L 2 (R, µ) is an unknown function. For example, if X is a moving average process, for some weight g ∈ L 1 (R), then we have is the inverse Fourier transform of g. Indeed, it is easy to show that every process X that is jointly wide-sense stationary with B can be represented as a sum of two wide-sense stationary processes, where X 1 is a process of the class (2) and X 2 (t) is uncorrelated with B(s) for all s, t ∈ R. It follows that and reduction to the class (2) is justified. Next, if a process X has the form (2) and for some positive integer m, then the m-th mean square derivative of X exists and admits a representation Hence, Now our problem (1) can be reformulated analytically as Notice that one can consider this problem with more or less arbitrary function ℓ(·) instead of a polynomial.
The spectral condition equivalent to adaptive setting is This condition clearly holds for all t ∈ R iff it holds for t = 0, i.e.

Discrete time, stationary sequence
Let (B(t)) t∈Z be a centered wide-sense stationary sequence, which means that E |B(t)| 2 < ∞ and its covariance function depends only on the time difference: According to the Herglotz theorem, K B (·) admits a spectral representation where µ is a finite measure on T := [−π, π) called the spectral measure of B. The sequence B itself admits a spectral representation where W(du) is a complex centered random measure with uncorrelated values on T controlled by µ. As in (2), we search an approximating sequence (X(t)) t∈Z , in the form where g(·) ∈ L 2 (T, µ). For example, if X is a moving average sequence, for some summable weight g, then we have is the inverse Fourier transform of g.
In the discrete case the notion of energy should be modified by replacing the (right) derivatives with their discrete analogues, e.g. X(t + 1) − X(t) for X ′ (t), X(t + 2) − 2X(t + 1) + X(t) for X ′′ (t), etc. Therefore, the instant energy of X takes the form The discrete-time version of problem (4) becomes Again, one can also consider this problem with arbitrary function ℓ(·) instead of the polynomial. The discrete-time analogue of kinetic energy corresponds to the increment α(X(t + 1) − X(t)), i.e. to the polynomial ℓ(z) = α(z − 1). One can consider the problem (5) either in the non-adaptive setting, or in the adaptive setting by requiring additionally g ∈ G ≤0 := span{e iτ u , τ ≤ 0, τ ∈ Z L 2 (T, µ)}.
2 First step: solution of the non-adaptive problem 2.1 Continuous time As we have seen for continuous-time setting, our problem states in (4) as For any complex numbers g and ℓ we have an identity Therefore, in the non-adaptive setting, where no further restrictions are imposed on the function g, the solution to (6) is given by the function depending on the energy form ℓ but not on the spectral measure µ. The minimum in (6) is equal to It is natural to call this quantity non-adaptive approximation error. In the control theory the term problem cost is also used.
In the simplest case of the kinetic energy ℓ(z) = αz, where α > 0 is a scaling parameter, we get Since g * (u) = 1 α 2 u 2 +1 is the inverse Fourier transform for we conclude that the solution to non-adaptive problem with kinetic energy for stationary processes is given, as suggested in (3), by the moving average process Notice that this solution is indeed non-adaptive because the future values of B are involved into approximation. The formula (8) was obtained in [5], see also [6] for the case of kinetic energy.
However, if adaptivity restriction is imposed on g, then (8) does not apply and we have to minimize the spectrum-dependent integral. By (7), the problem (6) reduces to This minimum (taken over g ∈ G ≤0 ) will be called additional adaptivity error and denoted ERR + A . This is the price we must pay for not knowing the future. The total approximation error, i.e. the minimum in (6) over g ∈ G ≤0 is then equal to These formulae were obtained in [5].

Discrete time
In the discrete-time setting, the situation is completely similar because the problem (5) differs from (6) only by replacing the spectral domain R with T and ℓ(iu) with ℓ(e iu ). Therefore, we obtain the expression for the nonadaptive error attained by the minimizer In the simplest case of the discrete-time kinetic energy ℓ(z) = α(z − 1), we have where The analogue of (10) is 3 Solutions to adaptive approximation problem Recall that adaptive approximation problem for continuous-time processes was reduced in (11) to solving the problem . This looks very much as a classical prediction problem, except for the function to be approximated: in our setting it is 1 |ℓ(iu)| 2 +1 , while in prediction problem it is e iτ u for some τ > 0. Therefore we may either directly reduce the approximation problem to the prediction problem, or to use methods that are usually used for solving the prediction problems. The latter way seems to be more efficient and general.

Continuous time
Consider continuous-time setting. Assume that we have an appropriate (to be made precise a bit later) factorization Then the left-hand side of (15) becomes Recall that the classical prediction problem is denote the solution of this problem. The solution of the general prediction problem with some finite complex measure ν ℓ depending on the energy polynomial ℓ(·), then the function It also follows from representation (19) that for any g ∈ G ≤0 we have 1 λ ℓ g ∈ G ≤0 . Now we impose another assumption on the factorization (16): If conditions (19) and (20) are verified, then the function minimizes (17) over g ∈ G ≤0 and thus solves the problem (15). Indeed, let g ∈ G ≤0 . If λ ℓ g ∈ L 2 (R, µ), then expression (17) is infinite because the bounded function 1/λ ℓ belongs to L 2 (R, µ). Let now λ ℓ g ∈ L 2 (R, µ). Then, using the optimality of q and the problem is solved. We stress that for polynomials a representation with properties (19) and (20) is always possible. Indeed, let ℓ(·) be a polynomial of degree M with complex coefficients. Then for all real u we have where P is a polynomial of degree 2M with real coefficients. Therefore, if β is a root of P, then β also is its root. Notice also that P has no real roots. Thus we may write where Im(β m ) > 0 and C > 0 (which follows by letting u = 0). Finally, we In this case the representation holds true and the existence of representation (19) for 1 λ ℓ follows. In order to verify (20), notice that, since the polynomial λ ℓ is bounded away from zero, condition λ ℓ g ∈ L 2 (R, µ) is equivalent to Using this fact with m = 1 we see that the functions converge to the function u g(u) in L 2 (R, µ) as δ → 0. Therefore, the limit also belongs to G ≤0 . One continues by induction and concludes that all functions u m g(u) for 1 ≤ m ≤ M belong to G ≤0 . Obviously, the same is true for their linear combination: λ ℓ g ∈ G ≤0 . For example, for continuous-time kinetic energy ℓ(z) = αz, we may use the factorization where λ ℓ (u) := 1 + iαu and

Discrete time
In the discrete-time setting, one should only replace R with T and ℓ(iu) with ℓ(e iu ) in (17). Now we need a factorization in place of (19), and satisfying an obvious analogue of (20), i.e.
If g ∈ G ≤0 and λ ℓ g ∈ L 2 (T, µ), then one must have λ ℓ g ∈ G ≤0 . (24) Then the function where the function q (τ,µ) * is the minimizer in the classical prediction problem, cf. (18), provides a solution for our problem.
We explain now how to construct the required factorizations for the typical forms of energy represented by arbitrary complex polynomials ℓ(·). p m z m is a polynomial of degree at most 2M with coefficients satisfying Hermitian symmetry condition p 2M −m = p m . Due to this symmetry, if β = 0 is a root of P, then 1/β also is its root. Notice also that P has no roots on the unit circle. Assume temporarily that ℓ 0 = 0. Then p 0 = ℓ 0 ℓ M = 0, hence zero is not a root of P and we may write for some complex C and |β m | > 1. Letting, say, z = 1, shows that the exterior constant is positive: Hence, we have the factorization The proof of the required properties is the same as in the continuous-time case. It is therefore omitted. Finally, notice that the temporary assumption ℓ 0 = 0 may be easily dropped. Indeed, in the general case we may always write ℓ(z) = z k ℓ(z) with some k ≤ M and ℓ(0) = 0. Then the factorization for ℓ also applies to ℓ because 1 + |ℓ(·)| 2 = 1 + | ℓ(·)| 2 on the unit circle.
For discrete-time kinetic energy ℓ(z) = α(z − 1), we may use a factorization and β from (13). In this case we see that holds as a version of (23).

Discrete time
We first recall few notions used in the analytical prediction technique. Let where Λ denotes Lebesgue measure, be the spaces of spectrally negative and spectrally positive functions. We will need a special class of outer functions. We do not recall the direct formal definition of an outer function, cf. [7, p.342]; instead, we use the following characterization, cf. [7, Theorem 17.23]: a function γ ∈ L 2 (T, Λ) is a conjugated outer function iff We stress that these functions are complex conjugated to outer functions as defined in Rudin [7]. In the sequel, however, we call them simply "outer functions"; this omission should not lead to any misunderstanding. Now we pass to the optimization problem. By (11), we have to compute Assume that the spectral measure has a density on T satisfying Kolmogorov's regularity condition, i.e. µ(du) = f (u)du and The classical prediction technique suggests to find factorizations with γ f being an outer function and λ ℓ , as above, satisfying conditions (23) and (24). Notice that assumption (29) implies the existence of factorization (30), cf. [7, Theorem 17.16]. Factorization (31) in the case of polynomial ℓ was given in (25).
Theorem 3.1 Let Q >0 be the orthogonal projection of γ f /λ ℓ onto L 2 >0 in the Hilbert space L 2 (T, Λ). Then the optimal adaptive approximation is given by X(t) = T g * (u)e itu W(du) with

The error of the adaptive approximation is given by ERR
Proof: We have = min Consider arbitrary g ∈ G ≤0 . Without loss of generality we may assume that λ ℓ g ∈ L 2 (T, µ) (otherwise the integral in (32) is infinite). Then by (24) we have λ ℓ g ∈ G ≤0 , which is equivalent to where the latter equality holds by (28) because γ f is an outer function.
For the discrete-time kinetic energy ℓ(z) = α(z − 1) we may proceed further as follows by using decomposition (26). Since γ f is an outer function, it belongs to L 2 ≤0 . By taking the Fourier series expansion and multiplying (27) and (35) we obtain Hence, By (33) and (36), it follows that where we used the identity We also have from (34) and (37) In the case when ℓ is an arbitrary polynomial, we can use (25) to construct a partial fraction decomposition of 1/λ l into a linear combination of fractions of the form 1/(1 − e iu β −1 m ) provided the numbers β m are pairwise distinct. Then we can apply the above considerations to every fraction separately.
It is possible to provide an explicit formula for the outer function γ f and the constant K in terms of the spectral density f . To this end, consider the function By [7,Theorem 17.16], the radial limits of its absolute value |q| are Lebesguea.e. given by lim Since the function √ f is square integrable, [7,Theorem 17.16(c)] implies that the function q belongs to the Hardy space H 2 on the unit disc. Defining we clearly have |γ f (u)| 2 = f (u) for u ∈ T. Also, γ f is a (complex conjugate of an) outer function by [7,Definition 17.14]. The Fourier series representation of γ f given in (35) translates into a Taylor series representation of q as follows: Returning to the case ℓ(z) = α(z − 1), it follows from (38) that Recalling (39) and doing straightforward transformations, we arrive at the following Theorem 3.2 In the discrete-time case with ℓ(z) = α(z − 1), the additional adaptivity error is given by where β = (2α 2 + 1 + √ 1 + 4α 2 )/(2α 2 ).
In the above argument, we assumed that T | ln f (u)| du < ∞, but (42) remains valid even when T | ln f (u)| du = ∞. Indeed, since f is a density, the latter condition is equivalent to and (42) states that ERR + A = 0. This result is easy to explain: It is known (e.g., [4, pp. 48-50]) that under (43) (or if f does not exist at all, see [4, Corollary 1 on p. 46]), it is possible to predict the future of the process X on the basis of its past perfectly, so that there is no difference between the non-adaptive and the adaptive approximation.
Let us look at our approximation problem when α ↓ 0 which means that we give less importance to the kinetic energy of the approximating process compared to the closeness of the processes. As α ↓ 0 we have β = α −2 + O(1) → ∞, and (42) yields The right-hand side looks very much like the classical Kolmogorov formula. Let us explain this similarity. Recall that the classical prediction problem asks to predict B(τ ) on the basis of B(0), B(−1), . . ., for τ = 1, 2, . . .. In particular, in the case of one-step prediction τ = 1, the Kolmogorov formula, see [2,Theorem 5.8.1] or [4, pp. 48-50], states that the mean square error of the optimal prediction is given by By (12), the optimal non-adaptive strategy is given by For the optimal adaptive strategy, it is therefore natural to make the ansatz where w is some function from L 2 ≤0 . The additional adaptivity error ERR + A is then Thus, the function w should be chosen as the solution to the classical prediction problem and we should have ERR + A ∼ α 4 σ 2 pred . This explains the similarity between (44) and Kolmogorov's formula. Observe, finally, that by (14), as α ↓ 0. Thus, for small α the price for not knowing the future is small compared to the error of the non-adaptive approximation.

Continuous time
The approach and the result is very much the same as for stationary sequences except for some integrability issues. We only replace T with R and redefine the spaces L 2 ≤0 and L 2 >0 in L 2 (R, Λ) as the spaces of Fourier transforms of functions supported by R − and R + , respectively.
Again we use the class of outer functions, this time with respect to the lower half-plane, cf. [3, p.36] and use the following characterization, cf. [3, p.39]: a function γ ∈ L 2 (R, Λ) is an outer function for the lower half-plane iff The Kolmogorov regularity condition now looks as follows: This condition ensures the existence of the factorization with γ f being an outer function, cf. [3, p.38]. For the energy function ℓ(·) we need a factorization with λ ℓ satisfying properties (19) and (20). It was shown in (21) above how to construct such factorization for polynomials. Now the construction of the optimal adaptive approximation and the calculation of the approximation error are done exactly as in the discretetime case but we repeat the approach for completeness of exposition. Theorem 3.3 Let Q >0 be the orthogonal projection of γ f /λ ℓ onto L 2 >0 in the Hilbert space L 2 (R, Λ). Then the optimal adaptive approximation is given by X(t) = R g * (u)e itu W(du) with

The error of the adaptive approximation is given by ERR
Proof: We have to compute where G ≤0 := span{e iτ u , τ ≤ 0 L 2 (R, µ)}.
By using factorizations, we have Consider arbitrary g ∈ G ≤0 . Without loss of generality we may assume that λ ℓ g ∈ L 2 (R, µ) (otherwise the integral in (47) is infinite). Then by (20) we have λ ℓ g ∈ G ≤0 , which is equivalent to where the latter equality holds by (45) because γ f is an outer function.
On the other hand, since γ f ∈ L 2 (R, Λ) and |λ ℓ | ≥ 1, we have Consider the unique orthogonal decomposition in L 2 (R, Λ) with Q ≤0 ∈ L 2 ≤0 and Q >0 ∈ L 2 >0 . Due to the orthogonality of the spaces L 2 ≤0 and L 2 >0 , we clearly have Furthermore, the equality is attained whenever It remains to prove that g * ∈ G ≤0 . Since γ f is an outer function, we have by (45) Q ≤0 ∈ L 2 ≤0 = span{γ f e iτ u , τ ≤ 0 L 2 (R, Λ)}, which is equivalent to Finally, we obtain from (19) the required inclusion thus completing the proof.
For continuous-time kinetic energy ℓ(z) = αz, by taking the Fourier integral representation and multiplying (22) and (50), we obtain Hence, By (48) and (51), it follows that Furthermore, by using (49) and (51), we obtain the continuous-time analogue of (41), To derive an explicit formula for the outer function γ f and the constant K in terms of the spectral density f , consider the function It is known [3, p. 37] that q(z) belongs to the Hardy space H 2 on the upper half-plane and the boundary limits of the absolute value |q| are Lebesgue-a.e. given by lim Defining we evidently have |γ f (u)| 2 = f (u) for u ∈ R. Further, γ f satisfies (45) by [3, p. 37, p. 39]. The Fourier representation of γ f , see (50), continues to hold in the upper half-plane: It follows from (52) that Recalling (53), we arrive at the following Theorem 3.4 In the continuous-time case with ℓ(z) = αz, the additional adaptivity error is given by As in the discrete-time case, if the Kolmogorov condition (46) is violated (or if the spectral measure µ does not possess a density at all), the perfect prediction of the future is possible and we therefore have ERR + A = 0.

Examples of adaptive least energy approximations
Unless the opposite is stated explicitly, in the following examples we consider kinetic energy, i.e. we let ℓ(z) = αz for continuous time and ℓ(z) = α(z − 1) for discrete time. Here α > 0 is a fixed scaling parameter.

Autoregressive sequence
A sequence of complex random variables (B(t)) t∈Z is called autoregressive, if it satisfies the equation B(t) = ρB(t − 1) + ξ(t), where |ρ| < 1 and (ξ(t)) t∈Z is a sequence of centered non-correlated complex random variables with σ 2 := E |ξ(t)| 2 not depending on t. In this case we have a representation For uncorrelated sequence we have a spectral representation where W is a complex centered random measure with uncorrelated values on T controlled by the normalized Lebesgue measure µ(du) := σ 2 du 2π . Therefore, we obtain We see that the spectral measure for B is which can be also found in Example 4.4.2 of [2]. By (56) and (14), the error of non-adaptive approximation equals to with β = β(α) defined in (13) (see [5] for detailed calculation). On the other hand, the spectral density factorizes as Hence, by (38) whereas by (39) Using (57) we conclude that From (41), (58), and (59), for the optimal prediction we have

Uncorrelated sequence
Consider an uncorrelated sequence as a special case of the autoregressive one with ρ = 0. The best adaptive approximation is given by and the approximation errors are Here we used again the identity (40) in the last step.