AttendLight: Universal Attention-Based Reinforcement Learning Model for
  Traffic Signal Control
  We propose AttendLight, an end-to-end Reinforcement Learning (RL) algorithm
for the problem of traffic signal control. Previous approaches for this problem
have the shortcoming that they require training for each new intersection with
a different structure or traffic flow distribution. AttendLight solves this
issue by training a single, universal model for intersections with any number
of roads, lanes, phases (possible signals), and traffic flow. To this end, we
propose a deep RL model which incorporates two attention models. The first
attention model is introduced to handle different numbers of roads-lanes; and
the second attention model is intended for enabling decision-making with any
number of phases in an intersection. As a result, our proposed model works for
any intersection configuration, as long as a similar configuration is
represented in the training set. Experiments were conducted with both synthetic
and real-world standard benchmark data-sets. The results we show cover
intersections with three or four approaching roads;
one-directional/bi-directional roads with one, two, and three lanes; different
number of phases; and different traffic flows. We consider two regimes: (i)
single-environment training, single-deployment, and (ii) multi-environment
training, multi-deployment. AttendLight outperforms both classical and other
RL-based approaches on all cases in both regimes.
