{'fraction_non_alphanumeric': 0.035830618892508145, 'fraction_numerical': 0.008686210640608035, 'mean_word_length': 5.122923588039867, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Standard error estimation by an automated blocking method

The sample mean X is probably the most popular estimator of the expected value in all sciences and var(X) measures the error (standardand mean-square-errors). Here, an alternative approach to estimation of var(X) for time series data is presented. The method has an accuracy similar to dependent bootstrapping, but scales in O(n) time, and applies to stationary time series, including stationary Markov chains. The computational complexity is bounded by 12n floating point operations, but this can be reduced to n + O(1) in large computations. Convergence in relative error squared is faster than n−1/2 and the method is insensitive to the probability distribution of the observations. It is proven that a small part of the correlation structure is relevant to the convergence rate of the method. From this, proof of the Blocking method [Flyvbjerg and Petersen, J. Chem. Phys. 91, 461 (1989)] follows as a corollary. The result is also used to propose a hypothesis test surveying the relevant part of the correlation structure. It yields a fully automatic method which is sufficiently robust to operate without supervision. An algorithm and sample code showing the implementation is available for PYTHON, C++, and R [www.github.com/computative/block]. Method validation using autoregressive AR(1) and AR(2) processes and physics applications is included. Method self-evaluation is provided by bias and mean-square-error statistics. The method is easily adapted to multithread applications and data larger than computing cluster memory, such as ultralong time series or data streams. This way, the paper provides a stringent and modern treatment of the Blocking method using rigorous linear algebra, multivariate probability theory, real analysis, and Fisherian statistical inference.