Multi-Vehicle Mixed-Reality Reinforcement Learning for Autonomous
  Multi-Lane Driving
  Autonomous driving promises to transform road transport. Multi-vehicle and
multi-lane scenarios, however, present unique challenges due to constrained
navigation and unpredictable vehicle interactions. Learning-based
methods---such as deep reinforcement learning---are emerging as a promising
approach to automatically design intelligent driving policies that can cope
with these challenges. Yet, the process of safely learning multi-vehicle
driving behaviours is hard: while collisions---and their near-avoidance---are
essential to the learning process, directly executing immature policies on
autonomous vehicles raises considerable safety concerns. In this article, we
present a safe and efficient framework that enables the learning of driving
policies for autonomous vehicles operating in a shared workspace, where the
absence of collisions cannot be guaranteed. Key to our learning procedure is a
sim2real approach that uses real-world online policy adaptation in a
mixed-reality setup, where other vehicles and static obstacles exist in the
virtual domain. This allows us to perform safe learning by simulating (and
learning from) collisions between the learning agent(s) and other objects in
virtual reality. Our results demonstrate that, after only a few runs in
mixed-reality, collisions are significantly reduced.
