Reinforcement Learning as Iterative and Amortised Inference
  There are several ways to categorise reinforcement learning (RL) algorithms,
such as either model-based or model-free, policy-based or planning-based,
on-policy or off-policy, and online or offline. Broad classification schemes
such as these help provide a unified perspective on disparate techniques and
can contextualise and guide the development of new algorithms. In this paper,
we utilise the control as inference framework to outline a novel classification
scheme based on amortised and iterative inference. We demonstrate that a wide
range of algorithms can be classified in this manner providing a fresh
perspective and highlighting a range of existing similarities. Moreover, we
show that taking this perspective allows us to identify parts of the
algorithmic design space which have been relatively unexplored, suggesting new
routes to innovative RL algorithms.
