Self-Awareness Safety of Deep Reinforcement Learning in Road Traffic
  Junction Driving
  Autonomous driving has been at the forefront of public interest, and a
pivotal debate to widespread concerns is safety in the transportation system.
Deep reinforcement learning (DRL) has been applied to autonomous driving to
provide solutions for obstacle avoidance. However, in a road traffic junction
scenario, the vehicle typically receives partial observations from the
transportation environment, while DRL needs to rely on long-term rewards to
train a reliable model by maximising the cumulative rewards, which may take the
risk when exploring new actions and returning either a positive reward or a
penalty in the case of collisions. Although safety concerns are usually
considered in the design of a reward function, they are not fully considered as
the critical metric to directly evaluate the effectiveness of DRL algorithms in
autonomous driving. In this study, we evaluated the safety performance of three
baseline DRL models (DQN, A2C, and PPO) and proposed a self-awareness module
from an attention mechanism for DRL to improve the safety evaluation for an
anomalous vehicle in a complex road traffic junction environment, such as
intersection and roundabout scenarios, based on four metrics: collision rate,
success rate, freezing rate, and total reward. Our two experimental results in
the training and testing phases revealed the baseline DRL with poor safety
performance, while our proposed self-awareness attention-DQN can significantly
improve the safety performance in intersection and roundabout scenarios.
