A Review of Safe Reinforcement Learning: Methods, Theory and
  Applications
  Reinforcement learning (RL) has achieved tremendous success in many complex
decision making tasks. When it comes to deploying RL in the real world, safety
concerns are usually raised, leading to a growing demand for safe RL
algorithms, such as in autonomous driving and robotics scenarios. While safety
control has a long history, the study of safe RL algorithms is still in the
early stages. To establish a good foundation for future research in this
thread, in this paper, we provide a review for safe RL from the perspectives of
methods, theory and applications. Firstly, we review the progress of safe RL
from five dimensions and come up with five problems that are crucial for safe
RL being deployed in real-world applications, coined as "2H3W". Secondly, we
analyze the theory and algorithm progress from the perspectives of answering
the "2H3W" problems. Then, the sample complexity of safe RL methods is reviewed
and discussed, followed by an introduction of the applications and benchmarks
of safe RL algorithms. Finally, we open the discussion of the challenging
problems in safe RL, hoping to inspire more future research on this thread.
  To advance the study of safe RL algorithms, we release a benchmark suite, an
open-sourced repository containing the implementations of major safe RL
algorithms, along with tutorials at the link:
https://github.com/chauncygu/Safe-Reinforcement-Learning-Baselines.git.
