Kernel autocovariance operators of stationary processes: Estimation and
  convergence
  We consider autocovariance operators of a stationary stochastic process on a
Polish space that is embedded into a reproducing kernel Hilbert space. We
investigate how empirical estimates of these operators converge along
realizations of the process under various conditions. In particular, we examine
ergodic and strongly mixing processes and prove several asymptotic results as
well as finite sample error bounds with a detailed analysis for the Gaussian
kernel. We provide applications of our theory in terms of consistency results
for kernel PCA with dependent data and the conditional mean embedding of
transition probabilities. Finally, we use our approach to examine the
nonparametric estimation of Markov transition operators and highlight how our
theory can give a consistency analysis for a large family of spectral analysis
methods including kernel-based dynamic mode decomposition.
