Control Theoretic Analysis of Temporal Difference Learning
  The goal of this paper is to investigate a control theoretic analysis of
linear stochastic iterative algorithm and temporal difference (TD) learning.
TD-learning is a linear stochastic iterative algorithm to estimate the value
function of a given policy for a Markov decision process, which is one of the
most popular and fundamental reinforcement learning algorithms. While there has
been a series of successful works in theoretical analysis of TD-learning, it
was not until recently that researchers found some guarantees on its
statistical efficiency. In this paper, we propose a control theoretic
finite-time analysis TD-learning, which exploits standard notions in linear
system control communities. Therefore, the proposed work provides additional
insights on TD-learning and reinforcement learning with simple concepts and
analysis tools in control theory.
