Learning from All Vehicles
  In this paper, we present a system to train driving policies from experiences
collected not just from the ego-vehicle, but all vehicles that it observes.
This system uses the behaviors of other agents to create more diverse driving
scenarios without collecting additional data. The main difficulty in learning
from other vehicles is that there is no sensor information. We use a set of
supervisory tasks to learn an intermediate representation that is invariant to
the viewpoint of the controlling vehicle. This not only provides a richer
signal at training time but also allows more complex reasoning during
inference. Learning how all vehicles drive helps predict their behavior at test
time and can avoid collisions. We evaluate this system in closed-loop driving
simulations. Our system outperforms all prior methods on the public CARLA
Leaderboard by a wide margin, improving driving score by 25 and route
completion rate by 24 points. Our method won the 2021 CARLA Autonomous Driving
challenge. Code and data are available at https://github.com/dotchen/LAV.
