Non-Markovian Reinforcement Learning using Fractional Dynamics
  Reinforcement learning (RL) is a technique to learn the control policy for an
agent that interacts with a stochastic environment. In any given state, the
agent takes some action, and the environment determines the probability
distribution over the next state as well as gives the agent some reward. Most
RL algorithms typically assume that the environment satisfies Markov
assumptions (i.e. the probability distribution over the next state depends only
on the current state). In this paper, we propose a model-based RL technique for
a system that has non-Markovian dynamics. Such environments are common in many
real-world applications such as in human physiology, biological systems,
material science, and population dynamics. Model-based RL (MBRL) techniques
typically try to simultaneously learn a model of the environment from the data,
as well as try to identify an optimal policy for the learned model. We propose
a technique where the non-Markovianity of the system is modeled through a
fractional dynamical system. We show that we can quantify the difference in the
performance of an MBRL algorithm that uses bounded horizon model predictive
control from the optimal policy. Finally, we demonstrate our proposed framework
on a pharmacokinetic model of human blood glucose dynamics and show that our
fractional models can capture distant correlations on real-world datasets.
