Reinforcement Learning for Traffic Signal Control: Comparison with
  Commercial Systems
  Recently, Intelligent Transportation Systems are leveraging the power of
increased sensory coverage and computing power to deliver data-intensive
solutions achieving higher levels of performance than traditional systems.
Within Traffic Signal Control (TSC), this has allowed the emergence of Machine
Learning (ML) based systems. Among this group, Reinforcement Learning (RL)
approaches have performed particularly well. Given the lack of industry
standards in ML for TSC, literature exploring RL often lacks comparison against
commercially available systems and straightforward formulations of how the
agents operate. Here we attempt to bridge that gap. We propose three different
architectures for TSC RL agents and compare them against the currently used
commercial systems MOVA, SurTrac and Cyclic controllers and provide pseudo-code
for them. The agents use variations of Deep Q-Learning and Actor Critic, using
states and rewards based on queue lengths. Their performance is compared in
across different map scenarios with variable demand, assessing them in terms of
the global delay and average queue length. We find that the RL-based systems
can significantly and consistently achieve lower delays when compared with
existing commercial systems.
