AttentionLight: Rethinking queue length and attention mechanism for
  traffic signal control
  Using Reinforcement learning (RL) techniques for traffic signal control (TSC)
is becoming increasingly popular. However, most RL-based TSC methods
concentrate on the RL model structure and easily ignore the traffic state
representation (vehicle number, queue length, waiting time, delay, etc.).
Moreover, some RL methods heavily depend on expert design for traffic signal
phase competition. In this paper, we rethink vehicles' queue length and
attention mechanism for TSC: (1) redesign the queue length (QL) as traffic
state representation and propose a TSC method called Max-QueueLength (M-QL)
based on our QL state; (2) develop a general RL- based TSC paradigm called
QL-XLight with QL as state and reward, and generate RL-based methods by our
QL-XLight directly based on existing traditional and latest RL models; (3)
propose a novel RL-based model AttentionLight base on QL-XLight that uses a
self-attention mechanism to capture the phase correlation, does not require
human knowledge on traffic signal phases' competition. Through comprehensive
experiments on multiple real-world datasets, we demonstrate that:(1) our M-QL
method outperforms the latest RL- based methods; (2) AttentionLight achieves a
new state-of-the-art (SOTA); (3) the state representation is essential for TSC
methods. Our code is released on Github.
