From Reinforcement Learning to Optimal Control: A unified framework for
  sequential decisions
  There are over 15 distinct communities that work in the general area of
sequential decisions and information, often referred to as decisions under
uncertainty or stochastic optimization. We focus on two of the most important
fields: stochastic optimal control, with its roots in deterministic optimal
control, and reinforcement learning, with its roots in Markov decision
processes. Building on prior work, we describe a unified framework that covers
all 15 different communities, and note the strong parallels with the modeling
framework of stochastic optimal control. By contrast, we make the case that the
modeling framework of reinforcement learning, inherited from discrete Markov
decision processes, is quite limited. Our framework (and that of stochastic
control) is based on the core problem of optimizing over policies. We describe
four classes of policies that we claim are universal, and show that each of
these two fields have, in their own way, evolved to include examples of each of
these four classes.
