Congested Urban Networks Tend to Be Insensitive to Signal Settings:
  Implications for Learning-Based Control
  This paper highlights several properties of large urban networks that can
have an impact on machine learning methods applied to traffic signal control.
In particular, we show that the average network flow tends to be independent of
the signal control policy as density increases. This property, which so far has
remained under the radar, implies that deep reinforcement learning (DRL)
methods becomes ineffective when trained under congested conditions, and might
explain DRL's limited success for traffic signal control. Our results apply to
all possible grid networks thanks to a parametrization based on two network
parameters: the ratio of the expected distance between consecutive traffic
lights to the expected green time, and the turning probability at
intersections. Networks with different parameters exhibit very different
responses to traffic signal control. Notably, we found that no control (i.e.
random policy) can be an effective control strategy for a surprisingly large
family of networks. The impact of the turning probability turned out to be very
significant both for baseline and for DRL policies. It also explains the loss
of symmetry observed for these policies, which is not captured by existing
theories that rely on corridor approximations without turns. Our findings also
suggest that supervised learning methods have enormous potential as they
require very little examples to produce excellent policies.
