SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for
  Autonomous Driving
  Multi-agent interaction is a fundamental aspect of autonomous driving in the
real world. Despite more than a decade of research and development, the problem
of how to competently interact with diverse road users in diverse scenarios
remains largely unsolved. Learning methods have much to offer towards solving
this problem. But they require a realistic multi-agent simulator that generates
diverse and competent driving interactions. To meet this need, we develop a
dedicated simulation platform called SMARTS (Scalable Multi-Agent RL Training
School). SMARTS supports the training, accumulation, and use of diverse
behavior models of road users. These are in turn used to create increasingly
more realistic and diverse interactions that enable deeper and broader research
on multi-agent interaction. In this paper, we describe the design goals of
SMARTS, explain its basic architecture and its key features, and illustrate its
use through concrete multi-agent experiments on interactive scenarios. We
open-source the SMARTS platform and the associated benchmark tasks and
evaluation metrics to encourage and empower research on multi-agent learning
for autonomous driving. Our code is available at
https://github.com/huawei-noah/SMARTS.
