Delay-Aware Model-Based Reinforcement Learning for Continuous Control
  Action delays degrade the performance of reinforcement learning in many
real-world systems. This paper proposes a formal definition of delay-aware
Markov Decision Process and proves it can be transformed into standard MDP with
augmented states using the Markov reward process. We develop a delay-aware
model-based reinforcement learning framework that can incorporate the
multi-step delay into the learned system models without learning effort.
Experiments with the Gym and MuJoCo platforms show that the proposed
delay-aware model-based algorithm is more efficient in training and
transferable between systems with various durations of delay compared with
off-policy model-free reinforcement learning methods. Codes available at:
https://github.com/baimingc/dambrl.
