Risk Sensitive Model-Based Reinforcement Learning using Uncertainty
  Guided Planning
  Identifying uncertainty and taking mitigating actions is crucial for safe and
trustworthy reinforcement learning agents, especially when deployed in
high-risk environments. In this paper, risk sensitivity is promoted in a
model-based reinforcement learning algorithm by exploiting the ability of a
bootstrap ensemble of dynamics models to estimate environment epistemic
uncertainty. We propose uncertainty guided cross-entropy method planning, which
penalises action sequences that result in high variance state predictions
during model rollouts, guiding the agent to known areas of the state space with
low uncertainty. Experiments display the ability for the agent to identify
uncertain regions of the state space during planning and to take actions that
maintain the agent within high confidence areas, without the requirement of
explicit constraints. The result is a reduction in the performance in terms of
attaining reward, displaying a trade-off between risk and return.
