Safer Autonomous Driving in a Stochastic, Partially-Observable
  Environment by Hierarchical Contingency Planning
  When learning to act in a stochastic, partially observable environment, an
intelligent agent should be prepared to anticipate a change in its belief of
the environment state, and be capable of adapting its actions on-the-fly to
changing conditions. As humans, we are able to form contingency plans when
learning a task with the explicit aim of being able to correct errors in the
initial control, and hence prove useful if ever there is a sudden change in our
perception of the environment which requires immediate corrective action. This
is especially the case for autonomous vehicles (AVs) navigating real-world
situations where safety is paramount, and a strong ability to react to a
changing belief about the environment is truly needed.
  In this paper we explore an end-to-end approach, from training to execution,
for learning robust contingency plans and combining them with a hierarchical
planner to obtain a robust agent policy in an autonomous navigation task where
other vehicles' behaviours are unknown, and the agent's belief about these
behaviours is subject to sudden, last-second change. We show that our approach
results in robust, safe behaviour in a partially observable, stochastic
environment, generalizing well over environment dynamics not seen during
training.
