Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning
  with Applications in Autonomous Driving
  Reinforcement learning (RL) can be used to create a decision-making agent for
autonomous driving. However, previous approaches provide only black-box
solutions, which do not offer information on how confident the agent is about
its decisions. An estimate of both the aleatoric and epistemic uncertainty of
the agent's decisions is fundamental for real-world applications of autonomous
driving. Therefore, this paper introduces the Ensemble Quantile Networks (EQN)
method, which combines distributional RL with an ensemble approach, to obtain a
complete uncertainty estimate. The distribution over returns is estimated by
learning its quantile function implicitly, which gives the aleatoric
uncertainty, whereas an ensemble of agents is trained on bootstrapped data to
provide a Bayesian estimation of the epistemic uncertainty. A criterion for
classifying which decisions that have an unacceptable uncertainty is also
introduced. The results show that the EQN method can balance risk and time
efficiency in different occluded intersection scenarios, by considering the
estimated aleatoric uncertainty. Furthermore, it is shown that the trained
agent can use the epistemic uncertainty information to identify situations that
the agent has not been trained for and thereby avoid making unfounded,
potentially dangerous, decisions outside of the training distribution.
