IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic
  Signal Control
  Scaling adaptive traffic-signal control involves dealing with combinatorial
state and action spaces. Multi-agent reinforcement learning attempts to address
this challenge by distributing control to specialized agents. However,
specialization hinders generalization and transferability, and the
computational graphs underlying neural-networks architectures -- dominating in
the multi-agent setting -- do not offer the flexibility to handle an arbitrary
number of entities which changes both between road networks, and over time as
vehicles traverse the network. We introduce Inductive Graph Reinforcement
Learning (IG-RL) based on graph-convolutional networks which adapts to the
structure of any road network, to learn detailed representations of
traffic-controllers and their surroundings. Our decentralized approach enables
learning of a transferable-adaptive-traffic-signal-control policy. After being
trained on an arbitrary set of road networks, our model can generalize to new
road networks, traffic distributions, and traffic regimes, with no additional
training and a constant number of parameters, enabling greater scalability
compared to prior methods. Furthermore, our approach can exploit the
granularity of available data by capturing the (dynamic) demand at both the
lane and the vehicle levels. The proposed method is tested on both road
networks and traffic settings never experienced during training. We compare
IG-RL to multi-agent reinforcement learning and domain-specific baselines. In
both synthetic road networks and in a larger experiment involving the control
of the 3,971 traffic signals of Manhattan, we show that different
instantiations of IG-RL outperform baselines.
