Finite Horizon Q-learning: Stability, Convergence, Simulations and an
  application on Smart Grids
  Q-learning is a popular reinforcement learning algorithm. This algorithm has
however been studied and analysed mainly in the infinite horizon setting. There
are several important applications which can be modeled in the framework of
finite horizon Markov decision processes. We develop a version of Q-learning
algorithm for finite horizon Markov decision processes (MDP) and provide a full
proof of its stability and convergence. Our analysis of stability and
convergence of finite horizon Q-learning is based entirely on the ordinary
differential equations (O.D.E) method. We also demonstrate the performance of
our algorithm on a setting of random MDP as well as on an application on smart
grids.
