{'fraction_non_alphanumeric': 0.03584102200141945, 'fraction_numerical': 0.01743891310960154, 'mean_word_length': 4.470604547975596, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Entropy test for complexity in chaotic time series

We perform an entropy test, combined with a diagnostic test for the degree of visible determinism, for the complexity in chaotic time series. A time series is coarse-grained into a binary sequence, partitioned into segments of D binary digits, and transformed into a string of “alphabets” binary-coded in D bits. Using the probability density function estimated from the histogram representing the frequencies of appearance of the 2 alphabets in the string, we calculate the information entropy referred to as the string entropy. Case studies are conducted for numerical time series generated by the logistic map, the tent map, the Lorenz equations, and the augmented Lorenz equations. We discuss the performances of the time series as sequences of pseudorandom numbers for chaotic cryptography.


Introduction
Information entropy, introduced by Shannon for analyzing problems in communication [1], has been applied to the time series analysis of chaotic dynamics [2,3]. Indeed, Fraser and Swinney showed that mutual information provides a practically useful criterion to determine the time lag for embedding, whereby embedding vectors are generated from a single-variate time series to reconstruct a phase space with the least redundancy in the information-theoretical sense [4].
The estimation of information entropy requires a probability density function that represents the uncertainty, i.e., the degree of disorder, in manifestations of chaotic dynamics. In time series analysis, the probability density function is often estimated from a histogram of observed data with a finite length. The data can be expressed in a manner that represents one's viewpoint on recognizing the dynamics underlying the time series. This allows us to construct a particular algorithm for estimating the information entropy in accordance with our way of recognizing the dynamical behavior.
For instance, Bandt and Pompe introduced a particular method for coarse-graining a time series, wherein the time series is transformed into a sequence of rank orders over sliding segments consisting of D data points, where D was referred to as the embedding dimension, to calculate the permutation entropy as a function of D using the probability density function of the rank order [5,6]. Another variant of the information entropy, referred to as the multiscale entropy, was devised by Costa et al. [7]. In their method, a time series is coarse-grained into a sequence of moving averages over sliding segments of data points sampled with time interval T . Then, the multiscale entropy as a function of T is estimated from the probability function of the moving average. Concerning the coarse-graining of time series using threshold-crossing, Bollt et al. published interesting results indicating that the choice of the threshold for mapping a time series to a sequence of symbols such as binary digits significantly affects estimates of the topological entropy, meaning that care must be taken to avert misdiagnosis of the dynamics underlying the time series [8].
Recently, we have proposed a chaotic cryptographic method using the augmented Lorenz equations [9], wherein the chaotic time series generated by the augmented Lorenz equations is used as a sequence of pseudorandom numbers to encrypt a message [10]. In our cryptographic method, a binary-coded plaintext is encrypted and decrypted by taking the logical operation of exclusive-OR between the plaintext and the binary-coded pseudorandom numbers transformed from the time series using coarse-graining with threshold-crossing. However, we have not shown how to evaluate the degree of complexity in the pseudorandom numbers, whereby we could guarantee the degree of security of our cryptographic method, because we were not aware of methods for evaluating the complexity suitable for our purpose, for instance, the entropy test published in [11], suitable for our purpose, for the complexity of pseudorandom numbers.
In this paper, we apply an information-theoretical method to evaluating the degree of complexity in a chaotic time series, which is essentially the same as the entropy test (National Institute of Standards and Technology, NIST 800-22) published in [11]. We resort to a particular type of information entropy, referred to as string entropy in this study, that may belong to a class of the topological entropy [8]. Although we are motivated to develop string entropy analysis because of our need to assess the performance of pseudorandom numbers for cryptosystems, it turns out that string entropy is useful for analyzing the dynamical properties of chaotic time series in that the probability density function used to estimate the entropy reveals frequent and/or missing symbols in the symbolic sequence transformed from the time series, which may be inherent in the chaotic dynamics. Furthermore, string entropy analysis will be shown to provide a useful criterion for selecting a time series as a sequence of pseudorandom numbers for cryptography when assisted by the algorithm developed by Wayland et al. [12] to estimate the degree of visible determinism in the time series. That is, the string entropy can evaluate the performance of pseudorandom numbers in terms of the degree of uniformity in random appearance of the alphabets generated from them, whereas the algorithm of Wayland et al. evaluates short-term predictability of a chaotic time series. These methods assess the different aspects of pseudorandom numbers, each of which provides important information to evaluate the entire performance of the random numbers.

Mathematical methods
The string entropy is defined and calculated by the following procedure. Given a time series with a sampling time interval of T (not explicitly shown in the notation), we coarse-grain x i into binary digits b i with the threshold crossing: b i = 0 if x i < x c and b i = 1 otherwise, where x c is an appropriately chosen threshold, e.g., the central value around which b i is distributed with equal probability. Next, the binary series {b i } N i=1 is partitioned into a sequence of Q segments consisting of D binary digits, where N = DQ. Then, each segment is mapped to the "alphabet" binary-coded in D bits. We thus obtain a string of Q alphabets as random realizations of a total of 2 D possible alphabets denoted as {a n } 2 D n=1 . When D = 6, for instance, we have 2 6 = 64 alphabets {'000 000', '000 001', . . . , '111 111'}, which correspond to {'0', '1' . . . , '63'} in the decimal expression and cover over the total number of the English alphabets. When D = 7, we have 2 7 = 128 alphabets as many as the total number of the ASCII codes. We count the frequency of appearance for each alphabet a n , with n running from 1 to 2 D , in the string of Q realizations and produce a histogram, from which the probability density function p(a n ) is estimated. Thus, the string entropy S is defined and estimated as p(a n ) log 2 p(a n ) . (1) To reduce the statistical error, Q should be sufficiently larger than 2 D , i.e., Q 2 D . The string entropy S takes the maximum value S max if and only if p(a n ) = 2 −D for n = 1, . . . , 2 D . Hence, S max = D [bits]. It is convenient to normalize S with respect to S max . Thus, the normalized string entropy H is defined as H = S/D, where 0 ≤ H ≤ 1 and H = 1 is obtained for completely random processes. Note that the (normalized) string entropy depends on the sampling time T of the original time series, although this is not explicitly shown.
String entropy analysis is reinforced when it is assisted by the algorithm developed by Wayland et al. [12] to measure the degree of visible determinism in time series. This algorithm is briefly summarized as follows. Suppose that we reconstruct a phase space by embedding from a given time series. In such a phase space, dynamical behavior is represented by trajectories consisting of embedding vectors. Then, the determinism underlying the time series can be recognized by the presence of nearby trajectories that point in similar directions, because the determinism governing the (chaotic) time evolution of a system exhibits similar behaviors in the past that cause similar behaviors in the (near) future. That is, the degree of visible determinism can be measured in terms of the diversity in the directions of neighboring trajectories in the phase space by the following procedure.
From the time series , where d ≥ 2 denotes the embedding dimension. We next randomly select a vector x p(0) and find its L nearest neighbors x p(l) with l running from 1 to L. We generate images of x p(l) denoted as x p(l)+τ for l = 0, . . . , L under an appropriate time interval τ . Then, the diversity of nearby trajectories is measured in terms of the translation error E trans defined by The more visible determinism there is in the time series, the closer E trans will be to zero. A previous numerical study [13] showed that time series can be regarded as (noisy) deterministic (chaotic) processes if E trans < 0.5 and as correlated stochastic processes if E trans > 0.5 [13]. In particular, uncorrelated stochastic processes yield E trans ≈ 1 independent of the embedding dimension. To reduce the statistical error, we take the mean over W medians of E trans for W sets of P randomly chosen x p(0) .

Numerical analysis and discussion
In this study, all numerical calculations were performed in double precision on a 32-bit machine. No particular methods were used to reduce the accumulation of roundoff errors. For the influence of finite precision on numerical chaotic time series, see [14]. Our first case study is concerned with chaotic maps, i.e, the logistic map and the tent map. Here, the sampling time T is not optimized and set to unity, because the chaotic maps are not supposed to be used as the pseudorandom-number generators in our cryptographic method [10] because of the security of secret key. When using the initial conditions of the maps as the secret key, the dynamical nature of the basin of attraction must be completely known. For the security of secret key, see [10] and the references cited therein. We use these chaotic maps to exemplify the performance of our method. For the optimal sampling time interval for the logistic map, see [14].
We estimate the string entropy for the logistic map x i+1 = αx i (1 − x i ), where 0 ≤ x i ≤ 1, and the parameter α is set to α = 3.95, 3.99, and 4. Figures 1(a), (b), and (c) show a typical example of the histograms for three realizations under randomly selected initial conditions of the logistic maps with α = 3.95, 3.99, and 4, respectively. Here, D = 6, N = 120000, Q = 20000, T = 1 (one time step), x c = 0.5 (that was chosen on the basis of the symmetry of the governing equation), and the initial 5000 data points (within which the autocorrelation function as well as the mutual information rapidly   Table I. At α = 3.95, there are missing alphabets, as indicated in Fig. 1(a). At α = 3.99, there still are a few missing alphabets in addition to frequently appearing alphabets, and H increases. At α = 4, there are no missing alphabets and all alphabets appear with approximately equal probability, which causes H to approach unity. Similar results for the tent map are shown in Fig. 1(d) and Table I. The tent map is defined as  Q = 20000, T = 1 (one time step), x c = 0.5, and the initial 5000 data points were eliminated from the analysis. In contrast to the logistic maps, there are many missing alphabets, whereas the alphabet '000 000' ('0' in the decimal expression) appears very frequently. Estimates of H are considerably smaller than those for the logistic maps.
As our second case study, we estimate the string entropy for the Lorenz equations [15] defined as a three-dimensional system of ordinary differential equations:ẋ = σ(y − x),ẏ = rx − y − xz, z = −βz + xy, where the dimensionless parameters σ, r, and β are set to σ = 10, r = 28, and β = 8/3, andẋ,ẏ, andż represent the derivatives of x, y, and z with respect to dimensionless time, respectively. The equations were numerically integrated using the 4th-order Runge-Kutta method with a time width of Δt = 0.01 under three sets of randomly chosen initial conditions (Gaussian random numbers with mean 0 and variance 1) of x, y, and z. The initial 5000 numerical solutions were eliminated from the analysis. Time series of x and y were obtained by discrete sampling of the solutions with a sampling time of T = 1 (T/Δt = 100 time steps). The sampling time was chosen so as to achieve a large value of H, as will be shown later. We mapped the time series to the corresponding binary series with x c = 0, around which the binary data are distributed with equal probability.  Table I. For both x and y, there are no missing alphabets and the three alphabets '000 000' ('0'), '011 111' ('31'), and '111 111' ('63') appear more frequently than other alphabets. The frequent appearances of '000 000' and '111 111' are considered to reflect random switchings of the sense of rotation, between clockwise and counterclockwise, of the convection roll in turbulent thermal hydrodynamic flows, which can be simulated by the Lorenz model [15]. Estimates of H for x and y are close to unity but smaller than that for the logistic map with α = 4.
Our third case study is concerned with the augmented Lorenz equations [9,10] defined aṡ where X is a dimensionless scalar variable, Y and Z are dimensionless K ×K diagonal matrices whose diagonal components are labeled as Y n and Z n , respectively, with n running from 1 to K: where the off-diagonal components of Y and Z are all set to zero.Ẋ,Ẏ, andŻ are the firstorder derivatives of X, Y, and Z with respect to dimensionless time, respectively, tr(·) represents the diagonal sum of a matrix, σ and R 0 are dimensionless parameters corresponding to the Prandtl and reduced Rayleigh numbers, respectively, and M denotes the diagonal matrix given by M = diag(M 1 , . . . , M n , . . . , M K ) with M 1 = 1 and M 2 through M K randomly taking values of M n = n or M n = n + 0.5. M can be used as a secret key for cryptography [10]. The diagonal coefficient matrices Φ and W are defined as The augmented Lorenz equations correspond to a star network of K Lorenz subsystems sharing the variable X as the central node. That is, the augmented Lorenz model is a (2K + 1)-dimensional system of ordinary differential equations. The bifurcation parameters σ, R 0 , and φ are set to σ = 25, R 0 = 3185, and φ = 0.36 [rad]. These settings were verified to yield chaos in our previous studies [9,10]. The augmented Lorenz equations were numerically integrated in a similar way to the Lorenz equations, except with a time width of Δt = 4 × 10 −4 and K = 101 (corresponding to the size of the secret-key space of 2 100 , which is prohibitively large to break the key by a brute-force attack) under For the dynamical behaviors of X and Y 100 , see [9,10]. Estimates of H are summarized in Table I. For X, there are no missing alphabets and the alphabets '000 000' ('0') and '111 111' ('63') appear more frequently than other alphabets. Similarly to in the case of the Lorenz model, this is considered to reflect random switchings of the sense of rotation of a chaotic gas turbine [9], which may simulate the dynamical behavior of largescale circulation (mean wind) in turbulent Rayleigh-Bénard convection with high Rayleigh numbers exceeding 10 6 [16]. In contrast, for Y 100 , all alphabets appear with substantially equal probability. Estimates of H for X and Y 100 are close to unity. In particular, the value for Y 100 is as large as that for the logistic map with α = 4.
Our numerical results indicate that the string entropy as well as frequent and/or missing alphabets are capable of characterizing the dynamics underlying complex time series. In terms of chaotic cryptography, the string entropy of pseudorandom numbers should be as close as possible to its maximum value of H = 1, and there should be neither frequent nor missing alphabets in the string. Frequent and missing alphabets can be a clue to identifying the dynamics generating pseudorandom numbers. Thus, one might consider that our string entropy analysis suggests that the logistic map with α = 4 is as useful as the augmented Lorenz equations (generating the time series of Y 100 ) for chaotic cryptography. However, string entropy analysis is reinforced when it is assisted by the diagnostic algorithm developed by Wayland et al. for recognizing determinism in time series, and it turns out that the logistic map is not suitable as a generator of pseudorandom numbers, as demonstrated below. Figure 3 shows estimates of the translation error E trans as a function of the embedding dimension d for the time series (sampled at a sampling time of T = 1) generated by the logistic map with α = 4 and the Lorenz and augmented Lorenz models, where N = 1000, L = 4, τ = 5, W = 10, and P = 51. Determinism is clearly visible for the logistic map when d ≤ 4, while the increase in E trans for d ≥ 5 is due to false nearest neighbors generated in the estimation of E trans . For x and y of the Lorenz model, determinism is as invisible as in the case of correlated stochastic processes. Note that X and Y 100 of the augmented Lorenz model yield large translation errors, despite the deterministic dynamics. In particular, the translations errors of Y 100 are close to unity as if the time series represents an uncorrelated stochastic process.
One might think that the test for visible determinism can solely determine which time series is effective as a source of pseudorandom numbers. However, this is not the case. In fact, as shown in Fig. 3, the translation errors of x and y of the Lorenz model are considerably greater than that of the logistic map irrespectively of the embedding dimension, whereas there is an opposite tendency for the normalized string entropy, as shown in Table I.
Hence, the string entropy analysis reinforced by the estimation of the degree of visible determinism indicates that Y 100 of the augmented Lorenz model is the most suitable generator of pseudorandom numbers for cryptography. The logistic map with α = 4 is not suitable because of the visible determinism in the time series, i.e., short-term predictability. The remaining chaotic time series are characterized by frequent and/or missing alphabets, which can be a clue to identifying the dynamics yielding the series.

Conclusions
We have applied string entropy analysis to characterize the complexity in time series generated by well-known dynamical models and the augmented Lorenz model. Our numerical results have shown that the string entropy as well as frequent and/or missing alphabets in the string of binary sequences mapped from the original time series are capable of characterizing the dynamics governing the time series. String entropy analysis is also shown to be reinforced when combined with the estimation of the degree of visible determinism in the time series. The test for visible determinism provides little information about uniformity in the statistical distribution of pseudorandom numbers, whereas the string entropy provides little information about short-term predictability of chaotic sequences. These methods are complementary to each other.
The string entropy depends on the sampling time interval of a time series, although this is not explicitly shown in this study. We conjecture that the multiscale entropy analysis published in [7] may be applicable to string entropy analysis, which is an open question to be investigated in our future study. This study is supported by JSPS Grant-in-Aid for Scientific Research (C) No. 15K00353.