Behavior Planning For Connected Autonomous Vehicles Using Feedback Deep
  Reinforcement Learning
  With the development of communication technologies, connected autonomous
vehicles (CAVs) can share information with each other. We propose a novel
behavior planning method for CAVs to decide actions such as whether to change
lane or keep lane based on the observation and shared information from
neighbors, and to make sure that there exist corresponding control maneuvers
such as acceleration and steering angle to guarantee the safety of each
individual autonomous vehicle. We formulate this problem as a hybrid partially
observable Markov decision process (HPOMDP) to consider objectives such as
improving traffic flow efficiency and driving comfort and safety requirements.
The discrete state transition is determined by the proposed feedback deep
Q-learning algorithm using the feedback action from an underlying controller
based on control barrier functions. The feedback deep Q-learning algorithm we
design aims to solve the critical challenge of reinforcement learning (RL) in a
physical system: guaranteeing the safety of the system while the RL is
exploring the action space to increase the reward. We prove that our method
renders a forward invariant safe set for the continuous state physical dynamic
model of the system while the RL agent is learning. In experiments, our
behavior planning method can increase traffic flow and driving comfort compared
with the intelligent driving model (IDM). We also validate that our method
maintains safety during the learning process.
