Learning from non-irreducible Markov chains
  Most of the existing literature on supervised learning problems focuses on
the case when the training data set is drawn from an i.i.d. sample. However,
many practical supervised learning problems are characterized by temporal
dependence and strong correlation between the marginals of the data-generating
process, suggesting that the i.i.d. assumption is not always justified. This
problem has been already considered in the context of Markov chains satisfying
the Doeblin condition. This condition, among other things, implies that the
chain is not singular in its behavior, i.e. it is irreducible. In this article,
we focus on the case when the training data set is drawn from a not necessarily
irreducible Markov chain. Under the assumption that the chain is uniformly
ergodic with respect to the $\mathrm{L}^1$-Wasserstein distance, and certain
regularity assumptions on the hypothesis class and the state space of the
chain, we first obtain a uniform convergence result for the corresponding
sample error, and then we conclude learnability of the approximate sample error
minimization algorithm and find its generalization bounds. At the end, a
relative uniform convergence result for the sample error is also discussed.
