Deep Reinforcement Learning with Robust and Smooth Policy
  Deep reinforcement learning (RL) has achieved great empirical successes in
various domains. However, the large search space of neural networks requires a
large amount of data, which makes the current RL algorithms not sample
efficient. Motivated by the fact that many environments with continuous state
space have smooth transitions, we propose to learn a smooth policy that behaves
smoothly with respect to states. We develop a new framework -- \textbf{S}mooth
\textbf{R}egularized \textbf{R}einforcement \textbf{L}earning
($\textbf{SR}^2\textbf{L}$), where the policy is trained with
smoothness-inducing regularization. Such regularization effectively constrains
the search space, and enforces smoothness in the learned policy. Moreover, our
proposed framework can also improve the robustness of policy against
measurement error in the state space, and can be naturally extended to
distribubutionally robust setting. We apply the proposed framework to both
on-policy (TRPO) and off-policy algorithm (DDPG). Through extensive
experiments, we demonstrate that our method achieves improved sample efficiency
and robustness.
