Improving Robustness via Risk Averse Distributional Reinforcement
  Learning
  One major obstacle that precludes the success of reinforcement learning in
real-world applications is the lack of robustness, either to model
uncertainties or external disturbances, of the trained policies. Robustness is
critical when the policies are trained in simulations instead of real world
environment. In this work, we propose a risk-aware algorithm to learn robust
policies in order to bridge the gap between simulation training and real-world
implementation. Our algorithm is based on recently discovered distributional RL
framework. We incorporate CVaR risk measure in sample based distributional
policy gradients (SDPG) for learning risk-averse policies to achieve robustness
against a range of system disturbances. We validate the robustness of
risk-aware SDPG on multiple environments.
