{'fraction_non_alphanumeric': 0.0325909454153782, 'fraction_numerical': 0.013724332024599563, 'mean_word_length': 4.625879593432369, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A novel Topological Model for Nonlinear Analysis and Prediction for Observations with Recurring Patterns

The paper introduces a novel topological method for prediction and modeling for a nonlinear time--series that exhibit recurring patterns. According to the model, global manifold of the reconstructed state--space can be approximated by a few overlapping recurrence neighborhoods. The inherent redundancy structure of the delay embedding procedure and the property of recurrence are used to reduce the computational load, which is inevitable in nonlinear analysis. The modeling and prediction possibilities of the model are demonstrated using (i) a numerical data generated by a dynamical system: the Duffing oscillator and (ii) a real--life data: Electrocardiogram ECG data of a healthy human. A potential application of the proposed model is demonstrated for a multivariate cardiovascular data set that exhibits the property of recurrence. Real--time monitoring of cardiovascular signals are essential in clinical research and corruption of data are very common. It is a challenging task for a model to perform cognitive functions based on the contextual information, explicitly predicting gaps or loss of data and identifying noises in the physiological data. Paper concludes with an application of the proposed model in addressing some of these the issues.


I. INTRODUCTION
Chaos theory and topology in the past few decades have given many useful insights to understand data generated by nonlinear systems. In 1980, Packard et al. observed that the state-space of a nonlinear system could be reconstructed from a time-series generated by that system [1]. It opened up a possibility of associating geometrical and topological structures with observed data [1,2]. Later Takens and Mañé gave a proof and listed conditions to be met for any reconstruction to be diffeomorphic to the original state-space [3,4]. State-space reconstruction techniques are used to develop a set of equations or maps for a system, from an observed time-series for prediction or description of the dynamics [5][6][7]. The standard methods broadly fall into two categories: (i) Global methods that find equations valid for the entire state-space [5,8] and (ii) Atlas methods that develop local charts/atlas for small neighborhoods of the state-space [9][10][11].
Nonlinear signals often mimic random signals despite their deterministic origin as are non-periodic and non-stationary. Recurrence plots and Poincare Sections are some classic techniques used for nonlinear analysis [12][13][14]. Recurrence (the property that a typical trajectory of the system keeps on visiting the neighborhood of a particular state in the state-space) is a characteristic of nonlinear chaotic systems and identification of recurrence patterns is a prominent method for nonlinear data analysis [15,16]. This paper introduces a novel topological model based on atlas for analyzing a specific class of nonlinear time-series that exhibit the property of recurrence.
The problem of state-space reconstruction is defined as follows. Assume that some discrete measurements or observations S i ∈ R of a dynamical systemẊ = F (X) where X ∈ R d and S = H(X) are available in a finite time-interval (0, Γ). Given that the evolution func- Delay, derivative and filtered data-coordinate embeddings are standard methods that address this question. They build a global multidimensional state-space using either delay vectors [3,4], derivatives [7,17] or filtered data vectors [18,19] generated from observations [20].
Embedding methods assume that the data used for reconstruction is an 'observable' i.e., it contains information about all state-variables of the system. In principle, we can determine the observability of any observations iff the system equations are known [21,22]. When the system equations are unknown, which is most of the reality, it is still ambiguous how to determine the observability of any observation. In this scenario, validation of a model through state-space reconstruction relies on the availability of multiple observations of the system, existence of many possible models and comparisons of the derived models for further analytical investigations [23,24].
While global models look for equations for the entire reconstructed state-space [23], atlas models partitions the reconstructed space into neighborhoods within which the dynamics can be approximated by local constant predictors [9,25], weighted predictors or other linear polynomials [26]. These models are known to use a lot of CPU resources in searching for nearest neighborhoods in the embedded space. The demands for resources are directly proportional to the required embedding dimension [27]. The model proposed here drastically reduces the computational time for searching neighborhoods by utilizing the recurrence property of the time-series and the redundancy of delay-coordinate embedding.
In what follows, Section II revises delay-coordinate embedding as a tool for modeling a system based on a time-series generated by it. Section III proposes a topological model 'Atlas for Recurrence Neighborhood (ARN)' for a time-series that exhibits recurring patterns; the model is based on a generalization of Eckmann's Recurrence-plots, for identifying a recurring pattern in the time-series and its neighborhood of recurrence. Section IV details a parametrization scheme and the generation of equations for evolution and conjugacy maps across the neighborhoods. Section V presents the results of data analysis for a numerically generated data of Duffing oscillator under chaos. Section VI discusses the application of the method for a real-world data: a univariate ECG data of a healthy human. Section VII explores an application of the ARN model for prediction, for a multivariate cardiovascular data set. Concluding discussions are presented in Section VII.

II. DELAY-COORDINATE EMBEDDING AS A MODELING TOOL
Delay-coordinate embedding is a convenient (not unique) representation of the state-space using delay-coordinates generated from the data. The scheme of state-space reconstruction rely on an assumption of observability of the time-series (the observation being mathematically generic monitoring all degrees of freedom of the system, thus containing information about other state-variables). In principle, observability of any observation can be verified if generating equations are known [22]. However, an inability to meet the requirement of observability with acquired data in practice can affect the faithfulness of the reconstruction.
Assume that S i ∈ R are observations of a dynamical system (X, F ) where X ∈ R d anḋ X = F (X) such that, S i = H(X i ) where H : R P → R denotes an observation function. The embedding theorems establishes the existence of a faithful reconstruction of the original state-space in some R P (where P > 2d) using P-dimensional delay vectors of the form {S n , S n+τ , . . . , S n+(P −1)τ } made from observations, where P is the embedding dimension and τ is the time-delay. Thus, the delay map D : R d → R P is generically faithful to the original system state-space if the embedding dimension P > 2d, the true dimension of the system [3,4]. The theorem also assumes the availability of infinitely many observations in a perfect noise-free scenario. However, in practice, we deal with a finite amount of discrete data in a highly probable noisy scenario. In addition to that, we need to face more limiting factors such as, the system noise along with the difference in time-scales between different parts of the system and its effects on observations [28].
Many heuristic methods are available to estimate the embedding dimension P and timedelay τ from data, since the evolution function F and the dimension d of the system under study are unknown, in most of the reality [29]. One general approach for estimating embedding dimension from data is to increase the dimension until all false near neighbors (FNN) reduce to zero (if data is noise free) or to a minimum (if data is noisy). In general, our faith in false near neighbor (FNN) tests are inversely related to the noise present in the data [30].
FNN-estimates can be considered as a lower bound for the embedding dimension for reconstruction, as the attractor can be assumed to be unfolded by then [20]. Takens theorem does not specify an upper bound on embedding dimension, and using a dimensional estimate much higher than the actual value is a general practice to deal with under-sampling and noise in data. In general, one would prefer a lower dimensional model as it is economical regarding data compared to its high dimensional counterpart. Also, spurious instabilities are reported in global analysis when a system is embedded in a dimension higher than what is required [31].
The model proposed here uses a high estimate for embedding dimension P which is higher than that is suggested in literature [20], as aim of the model is to identify a neighborhood for a recurring pattern (of length P ) in a time-series using delay-coordinate embedding in R P . It uses an atlas-based procedure for partitioning the reconstructed space into some finite number of neighborhoods and approximating the dynamics in the neighborhoods using various maps [9,25,26]. We use a data-driven parametrization scheme for manifold learning by mapping the data from a high dimensional delay embedding space to a low dimensional parametric space. A recurrence neighborhood for the specific pattern is parameterized in R d .
We assume that neighbourhood lives in a thinner manifold of dimension d even though the data belongs to high dimensional space R P since d << P . Section V and VI deal with highly sampled pattern (800 data points), hence P used is 800, and section VII has under-sampled pattern (110 data points) in real-world data, hence P used is 110.
Time-delay τ can be estimated based on available rules of thumb [20,29]. The process of delay-coordinate embedding requires an identification of a basis to unfold the attractor.
One would prefer a set of basis vectors that are possibly independent of each other for this purpose. Since delay vectors by their structure carry much mutual information across them, the value of the smallest time-lag for which the mutual information is a minimum across the delay vectors is proposed as a reasonable delay for embedding [32]. However, in practice when the data is noisy, and possibly under-sampled, the mutual information curve may not show a minimum, so a delay of unity (τ = 1) is suggested [20]. We stick to the delay of unity as the proposed model is designed to benefit from the redundancy of information that could be available in the delay-vectors. Sampling frequency was known for the real-world data we considered for analysis. We set delay as unity for delay-embedding for two specific reasons (i) signals recorded were human physiological measurements which could be under-sampled and noisy (ii) the proposed method wants to exploits whatever mutual information that is present in the delay vectors. The redundancy of information between delay vectors is a significant factor that reduces the computational cost of data analysis.

III. NEIGHBORHOOD OF RECURRENCE IN DELAY-COORDINATE EMBED-DING
Recurrence was formally introduced in 1890 by Henry poincaré as a property of the system to recur as infinitely many times as close to an initial state [33]. Later in 1987, Eckmann et al. introduced Recurrence Plots to visualize the property of recurrence from a time-series of the system. A system is defined as recurrent if a time-series generated by it keeps on visiting a specific neighborhood of the manifold in the state-space [34]. Following this, analyzing recurrence patterns and finding them in a time-series is a prominent method for data analysis [15,16,35] in numerous fields [12,[36][37][38][39].

A. Eckmann's Recurrence Plots (RP) for visualization of recurrence
Recurrence plot (RP) provides a two-dimensional visual representation of a trajectory generated by a dynamical system. Consider a finite time-series {x n } 1=1...N that represents a trajectory of the system. Then RP is defined by a 2D matrix as follows: when R(i, j) is true, (i, j) are the pair of times at which the trajectory is in the same ǫ-neighborhood of the state-space, i.e., ( Visualization of RP, due to its rich geometrical structure is used as a tool to find the correlations that are present in different scales within the data. Further, a series of measures listed in Recurrence Quantification Analysis (RQA) can be used to define RP quantitatively [12,15].

B. Generalized Recurrence (GR) for identifying recurring patterns
In this section, we define a generalization for RP, to identify recurrence of a single chosen pattern in a time-series as follows. Let {y p } p=1...P be a finite sequence of length P , that represents a specific pattern in the given time-series {x n } n=1..N such that P ≪ N as depicted in Figure 1. Refer as an example, Figure 8 in 8(b). In order to identify the sequence {y p } p=1...P in the given series, construct a set This process is equivalent to the delay embedding of {x n } in R P with a delay 1 and dimension P , as Z ∈ R P [20].
Then the Generalized Recurrence function is defined as follows: when GR(i, j) is True, (i, j) are the pair of times at which the delay vectors Z i , Z j are significantly close to each other. Alternately, the points Z i , Z j of the trajectory of Z (evolution of P -dimensional delay vector) are in the some ǫ-neighborhood of the reconstructed represents an appropriate choice of norm of the difference vector in R P . Note that GR(i, j) is a function of P and it can capture recurrence of all possible sequences of length P in the given time-series {x}.

C. Neighborhood of Recurrence for a Specific Pattern
We have seen that GR(i, j) by definition can capture recurrence of all possible sequences of length P in the given time-series {x}. Let us focus on the recurrence of a specific pattern {y p } of length P , this pattern will be referred as Z ref henceforth. Define a similarity measure L for the reference pattern Z ref with respect to all other Z i ∈ R P , such that

IV. ATLAS FOR RECURRENCE NEIGHBORHOODS (ARN) MODEL AND PARAMETRIZATION SCHEME
Atlas methods focus on partitioning the embedded space into some finite number of neighborhoods {U m } M m=1 and then approximate the dynamics in the neighborhoods using various maps [9,25,26]. A practical difficulty of this approach is the computational cost for finding close neighbors that grows exponentially with embedding dimension [27]. Here we introduce an 'Atlas for Recurrence Neighborhood (ARN)' model specifically for analyzing a time-series that exhibit the property of recurrence. The redundancy structure of delay embedding along with the recurrence in time-series reduce the computational load that is required otherwise. The ARN model computes a set of neighborhood preserving embeddings for Recurrence Neighborhoods reconstructed from the data.
The ARN model is based on a data-driven parametrization scheme. It does manifold learning by mapping the data from a high dimensional delay embedding space to a low dimensional parametric space following an empirical procedure. We have seen how to construct a recurrence neighborhood RN for a specific pattern ( Since d << P , we assume that RN lives in a thinner manifold of dimension d even though the data belongs to high dimensional space R P . ARN model is used to approximate the dynamics on the manifold using lower order polynomial maps. The linear transformation A (of dimension d × P operates on an P × 1 vector of R P to generate a d × 1 vector of R d ) represents the projection: R P → R d .
Neighborhoods translated to the origin are: Define a conjugacy map T , across the neighborhood matrices rX and rY such that, Now rX of dimension q × P , rY of dimension q × d are known matrices; T of dimension d × N can be calculated by pre-multiplying rX with the generalized inverse of rY .
T represents a linear map across the neighborhoods rY, rX. Hence, for every vector α in RY there exists a vector β in RX such that, Once T and the centroid RX are known, a vector in R N can be generated from a vector in R d . The transformation T represents the mapping from R d → R N as any of the recurrence vectors in R N can be predicted using their R d counterparts.
T : R d → R N of Eq. 11 together with A : R N → R d of Eq. 6 represents the topological conjugacy across the recurring neighborhoods in R N and R d .

C. Evolution of Dynamics in R d using Equivalence Classes
The members of the recurrence neighborhood in R d can be further classified into different equivalence classes based on their recurrence cycle τ (find τ as suggested in section III C).
Collect all the vectors that recur at the same time to belong to one equivalence class.
Evolution maps for a particular equivalence class can be defined as follows.
Collect all beginning and end vectors of a specific recurrence cycle τ in R d to matrices SX, SY respectively. Assume SX, SY contain q vectors each. Let sX and sY represents the equivalence classes translated to the origin (the centroid of SX, SY subtracted from the neighbors to translate to the origin). Record the centroid of SX and SY as SX and SY respectively. let Neighborhoods translated to the origin are: sX q,j = SX q,j − SX j for all q, for j = 1, 2 . . . d (14) sY q,j = SY q,j − SY j for all q, for j = 1, 2 . . . d Define a transformation V, across the matrices sX and sY such that; If the specific equivalence class contains q members, sX, sY both of dimension q × d are known matrices; V of dimension d × d can be calculated by pre-multiplying sY with the generalized inverse of sX.
The transformation V is a function of the recurrence cycle τ associated with the chosen equivalence class. Once the V matrix is estimated, any random vector α ∈ SX can be evolved to β ∈ SY .
β represents the evolution of α at the end of the recurrence cycle τ in space R d . In this manner, one can evolve a vector in R d using the evolution maps for all the equivalence classes (for all recurrence cycles) along with corresponding centroids.  (c = 0.04496, k = 0, ρ = 1, α = 0, F = 1.03) for which the system exhibit chaos. Figure 2 represents a numerically generated state-space of the oscillator for a time-series generated from an initial condition (1.226, 0.868).
x + cẋ + kx + ρx 3 = F cos(ωt + α) Results presented in this section are specific for the parametrization of the neighborhood in R 3 and the prescribed model aims to approximate the recurrence neighborhood in R 800 by a 3-dimensional manifold. The neighborhood was approximated by a linear subspace and nonlinear manifold by minimizing the least squared errors using a standard and nonlinear Singular Value Decomposition [11].

C. Prediction Results by ARN Model
Once the recurrence neighborhood, recurrence cycles, and equivalence classes are identified in R 3 , prediction using conjugacy maps and evolution maps can be made as explained in section IV D. ARN models approximates the dynamics of the system by a few overlapping recurrence neighborhoods with specific maps for each of them. Results of prediction for the neighborhood vectors in R 800 using the parametrization maps in R 3 are presented here. Predicted signals were compared with the original signals using two scores Q1 (based on mean squared errors) and Q2 (based on correlation). Scoring Functions are given in Appendix A.
The Figures and scores given here are specific for the vectors that belong to the period-3 (τ = 1500) equivalence class. The period-3 equivalence class had 102 members for the numerical simulation under trial. Figure 5 displays the prediction P 1a, P 1b for two random members (indexed 50 th , 120 th in the recurrence neighborhood) that belong to the period-3 equivalence class: 5(a) P 1a is the prediction for 50 th member using the affine map, RX is the centroid fit and RX 50 is the original vector, 5(b) P 1b is the prediction for 120 th member using the affine map, RX is the centroid fit and RX 50 is the original vector, and 5(c),(d) displays the errors in prediction for the affine fits and the mean fits with respect to the original vectors. Predictions can be further improved by fitting a nonlinear manifold by adding quadratic terms to the Atlas. Figure 6 displays the improved predictions by adding nonlinear terms in the Atlas for the neighborhood. Improved scores for 50 th member were (Q1 = 0.99997, Q2 = 0.99999) and that for the 120 th member were (Q1 = 0.9998, Q2 = 0.9999). Figure  in beat-to-beat cycles of the ECG (Heart Rate Variability HRV) which can be seen as a variation in recurrent intervals is a typical characteristic of a healthy ECG measurement [40].
Since the signal exhibit both nonlinear and nonstationary features, both stochastic and nonlinear models are in use to capture essential aspects of the heart dynamics [41,42].

C. Prediction Results by ARN model
Predicted signals were compared with the original signals using two scores Q1(based on Mean Squared Errors) and Q2 (based on correlation). Figure 10 displays the prediction P 1a, P 1b for two random members (indexed 60 th , 150 th in the recurrence neighborhood RX) that belong to the equivalence class corresponding to τ = 695ms. Figure 10(a) P 1a is the prediction for 60 th member using the affine map, RX is the centroid fit. Similarly Predictions can be further improved by adding quadratic terms for the equation for the Atlas as we have seen in Section V. Figure 11 displays the improved predictions by adding nonlinear terms in the Atlas for the neighborhood. Improved Prediction scores  Figure 13 shows a reconstructed state-space of the ECG signal in R 2 and R 3 after a projection from the delay-coordinate space R P . The property that the data lie in a ribbonlike manifold is used for prediction. However, empirical estimates for correlation dimension from ECG data are known to be higher (in the range of {9. . . 15} than 2 or 3 [44]). However, the proposed model assumes that the dynamics heart can be seperated into two components: (i) the one that is responsible for the heart rate variability (ii) the effect of this variability on the pumping action of heart. The dynamics attributed to former 'the variability in recurrences' is possibly higher due to its stochastic/nonlinear nature. The model assumes that if the higher order dynamics behind the variability in recurrences are taken apart, rest of the heart dynamics is simple (with a lower dimensional structure) that involves the switching of the equivalence classes corresponding to the recurrence cycles. Once an optimal neighborhood is found, the prediction is straightforward using ARN model.
If the P value is chosen much lesser than RV min , one will have to consider multiple neighborhoods to cover the global reconstructed state-space. When P is lesser than the optimal value, vectors of the recurrent neighborhood in R P do not overlap the time-domain signal completely for the entire recurrent cycle resulting in a partial prediction of the signal.
In that case, multiple recurrent neighborhoods can be used to predict the entire structure, by patching up the partial predictions. Figure 14 shows partial predictions from 3 different recurrence neighborhoods for a blood pressure signal ABP from ECGII in c21.dat [45] for P = 30, whereas the optimal value P = 110 gave full predictions.

VII. MULTICHANNEL DATA-PREDICTION ACROSS CHANNELS
This section explores capabilities of ARN model to analyze an extensive data set of multichannel recordings of physiological signals provided by Physionet [46]. It contains three sets A, B, C each containing 100 multichannel cardiovascular recordings, which are 10 minute in length, sampled at the rate of 125 Hz. Signals in each of the multichannel recording vary across the records, and they include various ECG channels, blood pressure, respiration outputs. For a typical multichannel recording of length 10 minutes, the last 30

24
FIG. 14: Partial predictions Pred1 (i), Pred2 (ii) and Pred3 (iii) from 3 different recurrence neighborhoods for a blood pressure signal ABP from ECGII in c21.dat [45] for P = 30 (Choosing P = 110 gave full predictions for this special case, and refer Table IV  Prediction strategy is demonstrated in (b). Channel 2 contains the missing data segment for which a prediction is made using data available in channel 1. A1, B1 are the modeling windows across which a conjugacy map is identified, which is used for prediction for data in the window B2 from window A2.
seconds of a randomly chosen channel is replaced by a gap (a flat line signal) as shown in Figure 15(a). The goal of our study was to reconstruct the missing signal in each record using the information available in other channels using the ARN model and compare the efficiency of the predictions (with respect to the originals that is available [45]).  Figure 15(b) illustrates the algorithm for the ARN model. Channel 2 contains the missing data segment and data available in Channel 1 is used for the prediction. A1, B1 are the modeling windows across which a conjugacy map is identified, for the prediction of data in the window B2 using data in the window A2.
We infer from the analysis that the scores of reconstruction are dependent on the quality of the signals that are available on the channels. If the data available in the modeling and prediction windows are noisy or clipped, the predictions too were affected. The parameters of the model estimated based on the modeling windows were used for prediction. Though the signals are nonstationary and nonlinear, placing the modeling and prediction windows closer in time, the model assumes that the parameters are invariant across the windows. For a prediction of data in a 30s window, a data of length 30s ahead was used for the model.
Neural network based methods are the best among the current successful methods due to the development of novel training strategies [47,48], which can solve complex problems and give great predictions with intense training sessions [49][50][51]. ARN model proposed here used estimated parameters from the modeling window and the lack of training has drastically reduced the time of prediction. Other than the excellent prediction capacity for the signals, lack of training time is an advantage of the ARN method over neural network methods. Tables I-IV contain (Q1, Q2) scores for all ECG and Blood pressure signals in set C [45].
We find the efficacy of ARN models for predictions as excellent based on the scores, and we infer that the model could effectively reflect the changes in the cardiovascular system across the channels. The scores for rest of the signals of set C {PLETH, RESP, ICP, CVP} were not as good as the {ECG, BP} scores when the recurrent cycles were calculated with respect to the ECG signals [52]. Reasons for weak predictions could be: either there is no direct connection between the channels, or the connection was highly non-linear, or the model developed was inefficient to extract the connection. Since RESP and CVP have identical recurrence cycles with respect to each other (that is different from ECG, BP recurrence cycles), there is a possibility for improving scores by finding recurrent neighborhoods corresponding to their actual recurrent cycles.

VIII. CONCLUSIONS
We introduced Atlas for Recurrence neighborhood (ARN) Model-a novel topological method for prediction and modeling for a nonlinear time-series that exhibits recurring patterns. The paper demonstrated the model using (i) a data generated by a well studied is in general inevitable in nonlinear analysis was reduced as the model exploited the redundancy structure of the delay embedding procedure along with the property of recurrence.
Since corruption of cardiovascular signals are very common in real-time monitoring, potentials of the proposed ARN model to perform a challenging cognitive task-'the prediction of gaps or loss of data based on the contextual information' were explored in the study reported here, using a set of 100 multichannel cardiovascular recordings.

IX. ACKNOWLEDGMENTS
Sajini thanks Amit Apte for many constructive criticisms on topological modeling methods that has enhanced her understanding of the field. She thanks Nithin Nagaraj for numerous helpful discussions on this paper. She is grateful to ICTS-TIFR for providing a peaceful and motivating working environment during her survival of a major setback on health. She thanks National Board of Higher Mathematics, Department of Atomic Energy Note that Q1=1, Q2=1 implies 100% fit between the original and the predicted signal.