Multi-Agent Connected Autonomous Driving using Deep Reinforcement
  Learning
  The capability to learn and adapt to changes in the driving environment is
crucial for developing autonomous driving systems that are scalable beyond
geo-fenced operational design domains. Deep Reinforcement Learning (RL)
provides a promising and scalable framework for developing adaptive learning
based solutions. Deep RL methods usually model the problem as a (Partially
Observable) Markov Decision Process in which an agent acts in a stationary
environment to learn an optimal behavior policy. However, driving involves
complex interaction between multiple, intelligent (artificial or human) agents
in a highly non-stationary environment. In this paper, we propose the use of
Partially Observable Markov Games(POSG) for formulating the connected
autonomous driving problems with realistic assumptions. We provide a taxonomy
of multi-agent learning environments based on the nature of tasks, nature of
agents and the nature of the environment to help in categorizing various
autonomous driving problems that can be addressed under the proposed
formulation. As our main contributions, we provide MACAD-Gym, a Multi-Agent
Connected, Autonomous Driving agent learning platform for furthering research
in this direction. Our MACAD-Gym platform provides an extensible set of
Connected Autonomous Driving (CAD) simulation environments that enable the
research and development of Deep RL- based integrated sensing, perception,
planning and control algorithms for CAD systems with unlimited operational
design domain under realistic, multi-agent settings. We also share the
MACAD-Agents that were trained successfully using the MACAD-Gym platform to
learn control policies for multiple vehicle agents in a partially observable,
stop-sign controlled, 3-way urban intersection environment with raw (camera)
sensor observations.
