Vision Transformer for Learning Driving Policies in Complex Multi-Agent
  Environments
  Driving in a complex urban environment is a difficult task that requires a
complex decision policy. In order to make informed decisions, one needs to gain
an understanding of the long-range context and the importance of other
vehicles. In this work, we propose to use Vision Transformer (ViT) to learn a
driving policy in urban settings with birds-eye-view (BEV) input images. The
ViT network learns the global context of the scene more effectively than with
earlier proposed Convolutional Neural Networks (ConvNets). Furthermore, ViT's
attention mechanism helps to learn an attention map for the scene which allows
the ego car to determine which surrounding cars are important to its next
decision. We demonstrate that a DQN agent with a ViT backbone outperforms
baseline algorithms with ConvNet backbones pre-trained in various ways. In
particular, the proposed method helps reinforcement learning algorithms to
learn faster, with increased performance and less data than baselines.
