{'fraction_non_alphanumeric': 0.03415037179840264, 'fraction_numerical': 0.008468741393555494, 'mean_word_length': 4.563002680965147, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Dynamic-Mode Decomposition and Optimal Prediction

In the following work, using the Mori-Zwanzig formalism from statistical mechanics, we develop an extension of the Dynamic-Mode Decomposition algorithm to account for missing information and thereby produce data-driven optimal predictions.


Introduction
In recent years, beginning with the now seminal papers [1,2], a large body of literature has emerged around the modal decomposition technique known as the Dynamic-Mode Decomposition (DMD); see [3] for a comprehensive review of the literature, and the recent reviews in [4] and [5] for the most recent summaries and extensions. The primary advantage of DMD is that it is a completely model free data processing tool, allowing for the ready determination of results from arbitrarily complicated data sets. Moreover, in contrast to other methods such as Principle Component Analysis, the DMD also provides a ready means for generating models from measured data alone, thereby allowing for a means of sidestepping model development in cases where it would either be difficult or even impossible to do so. However, this advantage comes at the cost of introducing an infinite dimensional framework by way of determining the spectral properties of the affiliated Koopman operator [6]. Authors have, through using functional analytic ideas, been able to exploit this infinite dimensional structure to enhance the performance of the DMD [7,8]. However, it has been shown that these enhancements are difficult to apply in a systematic way [9], and thus approaches merging the DMD with nonlinear optimization are currently being explored to improve the accuracy and usability of the method [10,11].
In a very different direction, building on the now seminal work from the statistical mechanics community in [12] and [13], in [14,15,16], by way of what was called a Mori-Zwanzig Decomposition (MZD), the Koopman operator formalism was ultimately shown to be an especially effective means for accounting for missing information in measurements related to dynamical systems. In these works, formulas for memory kernels were derived whereby the impact of unsampled dimensions in a dynamical system could be accounted for in a statistically consistent way, thereby helping to better predict the correct averaged results one would expect if one had an essentially unlimited number of measurements. Given that this is never the case in practical situations, the MZD approach then provides an especially attractive means for approximating averaged dynamics in potentially complex dynamical systems.
Other authors have used this MZD approach to help generate more refined coarse-scale molecular dynamics simulations [17], close numerical schemes with respect to sub-grid scale variables [18], and help provide longtime accurate reduced order models for nonlinear dynamical systems [19]. Motivated by this, in this work we merge the Koopman operator based MZD method with the standard DMD algorithm to introduce memory dependence into the DMD algorithm. We term this new algorithm the Memory Dependent DMD (MDDMD). Using this, we are able to derive a modelindependent algorithm which allows from a single time series of data, representing only a partial measurement of a dynamical system, the accurate estimation of the reduced dynamics found by averaging over the missing or unmeasured dimensions. Our results are supported with numerical simulation, which show our method to be quite effective. Thus, we have derived a means for accurately predicting averaged dynamics from one partial time series without making recourse to any particular dynamical system or model. Likewise, as with the standard DMD, we are able to generate a corresponding model which could be used for prediction beyond the given time series.
The utility of such a feature will be explored in future work.
The rest of the paper is structured in the following way. The remainder of this section is devoted to an introduction to the Koopman operator, the standard DMD method, and the MZD method. Section 2 presents a derivation of our MDDMD algorithm and approximation with numerical results following. Section 3 presents our conclusions and discussions of future work. An Appendix summarizes technical results used throughout the paper.

Koopman Operators and the Dynamic-Mode Decomposition
We study nonlinear dynamical systems of the generic form where we suppose N s ≫ 1. We denote the affiliated flow of this equation as y(t) = ϕ(t; x). We define the associated Hilbert space of observables, say where µ is some appropriately chosen measure. A great deal of insight can be gained from looking at the affiliated linear representation of the problem given via the infinite-dimensional Koopman operator K t , where The power in this approach is that by moving to a linear-operator framework, the affiliated dynamics of the nonlinear system as measured via observables is captured via the eigenvalues of K t . Thus, assuming for the moment that K t has only discrete spectra, then if we can find a basis of L 2 R Ns , R, µ via the Koopman modes h j where K t h j = e tλ j h j then for any other observable g, we should have However, as one would imagine, determining the modes of the Koopman operator is in general impossible in an analytic way. The Dynamic-Mode Decomposition (DMD) method [2,1,7,3] has been developed which allows for the determination of a finite number of the Koopman modes.
To wit, if we sample the flow ϕ(t; x) at a discrete set of times t j , where j = 0, · · · , N T , thereby generating the data set y j = ϕ(t j , x), and if we select some set of observables g = {g l } M l=1 , then the DMD method approximates K δt by computing the spectra of the finite-dimensional operatorK a , which solves the the finite-dimensional least-squares problem in the Frobenius normK where G − and G + are the M × N T matrices of the past and future states respectively, i.e.
where we let Σ be strictly diagonal with strictly nonzero entries reflecting the rank of G − , then we have that where N r is the rank of G − and P u is the affiliated orthogonal projection onto the column space of U . Thus we can solve the minimization problem by lettingK − denotes the Moore-Penrose pseudoinverse. The efficacy of this approach though is contingent on the choice of observables one makes, and to date no general strategy for observable choosing strategies has been made. Some success has been had by trying to use kernel based means allowing for the efficient exploration of more complicated, nonlinear observables; see [8]. However, as shown in [9], for infinite dimensional problems, or at least the relatively high-dimensional dynamical systems which approximate them, kernel based techniques can be of very limited utility. By using various reproducing kernel techniques from machine learning, [20], more robust methods for determining effective observables can be found. While effective, the methodology in [20] is computationally intensive requiring the manipulation of large matrices.

The Liouville Equation and the Mori-Zwanzig Decomposition
Treating the nonlinear dynamical dynamical system as a vector field, we can form the affiliated Liouville equation which is an initial value problem of the form Thus we see that the Koopman operator is a stand in for the affiliated semi-group e Lt . To wit, ifφ We now suppose that there is an orthogonal projection P acting on L 2 R Ns , R, µ with corresponding complement Q = I − P. Through an approach first derived in [12] and [13] and generalized in [14], we are able to rewrite the given dynamical system or equivalent Liouville equation in the form and where K(t; g) is called the memory kernel, given by where the noise F (t; g) is given by the dynamics orthogonal to P so that Note, Equations (2) and (3), hereafter known as the Mori-Zwanzig decomposition (MZD) are not approximations, instead being an equivalent reformulation of the Liouville initial value problem. The focus of this paper is on discretizing the MZD in such a way as to only depend on knowledge of the observations g(ϕ t (x)) with no recourse made to L and thus an explicit model dynamical systemẏ = f (y).

Memory Dependent DMD: the DMD and Mori-Zwanzig Decomposition
We now suppose that a given observable g(x) is dependent on two different, orthogonal sets of coordinates so that g(x) = g(x,x), where we imagine that x represents those coordinates which are accessible to measurement whilex are not readily observable. Given an appropriate probability measure, sayp, over the unmeasurable variablesx, then for any observable g(x, t) we have a projection P via conditional expectations, i.e. Pg( Thus we suppose that our 'data' stream is given byĝ n ∈ C N d wherê g n (x) = Pe Ltn g, t n = nδt, i.e. all we can measure are the time dynamics of a process with regards to the observable dimensionsx. Likewise, we treat L and e tL as terms to discover via the data. Following the DMD paradigm, we definẽ Ultimately, we wish to find a finite-dimensional diagonalization ofT (δt) so that ifT Thus we have the consistent approximations In order to get a closed form approximation, in Equation (3), we further suppose that and we use a first-order-finite-difference approximation to the derivative. Note, we see that the approximation in Equation (4) is internally consistent with the assumptions we have made by seeing that Thus, using the expansion we readily get Equation (4). Note, this does not show that the DMD is somehow inexplicably accurate. Instead it shows that the DMD assumptions we have made prevent us from directly determining any aspect of the orthogonal dynamics associated with the projection Q.
Using the trapezoid method for the remaining integrals and letting we get that by using the spectral representation theorem for matrices, we get the extended, memory-dependent optimization problem where we havẽ This in turn gives us the nonlinear optimization problem for findingK a so that ExpandingK a =K a,0 + δtK a,1 + · · · , we readily get that Note, see the Appendix for details on how to approximate DM K a,0 ; K 0 for computational purposes. We now follow the standard DMD procedure and recover the traditional DMD decomposition by diagonalizingK a,0 so that Pursuing a perturbative result, we then look at the eigenvalue problem so that we find, assuming for now the simplicity of the unperturbed eigenvalue λ 0,j , that where N denotes the null-space of a matrix, and ·, · is the canonical inner product on C N d . This likewise allows for one to find the perturbation to the eigenvector via the formula Thus, treating K 0 as a random vector, we see that we have now generated an ensemble of approximations to the projected semi-groupT (δt) = Pe Lt . Averaging over these then gives us the approximatioñ where again N r is the rank ofK a,0 . Decomposing the stochastic terms into their means and fluctuations so that (v l,0 + δtv l,1 ) exp t δt λ 0,l − 1 + logλ 1,l b l .
This then gives us a formula for finding what we call the memory dependent DMD (MDDMD). We can summarize our algorithm in the following way.
1. Given data stream G − and G + , compute and diagonalize . This is the standard DMD algorithm.
2. From a given distribution, generate an ensemble of random vectors K 0 . Use these to computeλ j (K 0 ) and v (a) 0,j as in Equation (6). 3. Compute the averages of these perturbation parameters and use them to construct the MDDMD approximation given by Equation (7).

Example and Results
Following the choice of example in [14], we study the four-dimensional Hamiltonian system generated by the Hamiltonian H(y) where with conjugate pairs (y 1 (t), y 2 (t)) and (y 3 (t), y 4 (t)). This system describes two nonlinearly coupled oscillators. We treat the initial conditions y 1 (0) = x 1 and y 2 (0) = x 2 as known while the remaining initial conditions are considered unknown, and thusx = (x 1 , x 2 ) andx = (x 3 , x 4 ). Fixingx = (1, 0), we pickx so thatx j ∼ N (0, σ 2 ), where N (0, σ) denotes a mean zero Gaussian random variable with variance σ 2 . We then generate the reduced dynamical quantities by averaging over N ens = 10 4 randomly generated choices ofx. Likewise, to generate the results for the MDDMD, we average over an ensemble of 10 4 choices for K 0 where each component is likewise chosen from N (0, σ 2 ). The unperturbed data forming the data streams G − and G + comes from measuring only (y 1 (t), y 2 (t)) T from one of the random realizations used to generate the quantities in Equation (8). We run each simulation for t f = 20 units of time with a time step of δt = 10 −1 , which for simplicity we also take to be our sampling rate of the MDDMD. Setting σ = .5 generates the results seen in Figure 1. As can be seen, while not exact, the MDDMD is able to reproduce the decaying oscillatory nature of the projected dynamics. That this is done from a single random measurement which itself does not exhibit the decay properties seen makes the MDDMD results especially compelling. Doubling the standard deviation so that σ = 1 generates the results in Figure 2. While not as accurate, the MDDMD is still able to track much of the dynamics of the reduced dynamics, though the added stochasticity causes the underlying measurement used in the MDDMD to push the approximation out of phase with respect to the true reduced dynamics.

Conclusion and Future Directions
We have derived in this paper an extension to the original DMD algorithm, the MDDMD algorithm, which facilitates accounting for the effects of missing information. This is done by introducing a memory kernel in the vein of what has been done in the statistical mechanics literature. This allowed us then to develop a method which produces reasonable approximations to averaged dynamics from one incomplete measurement. This likewise allows us to generate a time-predictive model accounting for the impact of missing information in a dynamical system. While the use of our method on a toy problem was studied in this paper, given the relative success, it is a future project to use this in more complex models or on real world data. Likewise, while a reasonably self-consistent approximation scheme was used to derive the MDDMD approach, it is an especially interesting question to explore other approximations which facilitate better resolving the impact of orthogonal dynamics, such as in [19]. This is also a subject of future research.      where the 'memory kernel' K(s; g) is given by K(s; g) = PLF (s; g).