Dynamic Interaction-Aware Scene Understanding for Reinforcement Learning
  in Autonomous Driving
  The common pipeline in autonomous driving systems is highly modular and
includes a perception component which extracts lists of surrounding objects and
passes these lists to a high-level decision component. In this case, leveraging
the benefits of deep reinforcement learning for high-level decision making
requires special architectures to deal with multiple variable-length sequences
of different object types, such as vehicles, lanes or traffic signs. At the
same time, the architecture has to be able to cover interactions between
traffic participants in order to find the optimal action to be taken. In this
work, we propose the novel Deep Scenes architecture, that can learn complex
interaction-aware scene representations based on extensions of either 1) Deep
Sets or 2) Graph Convolutional Networks. We present the Graph-Q and DeepScene-Q
off-policy reinforcement learning algorithms, both outperforming
state-of-the-art methods in evaluations with the publicly available traffic
simulator SUMO.
