Dynamic Graph Configuration with Reinforcement Learning for Connected
  Autonomous Vehicle Trajectories
  Traditional traffic optimization solutions assume that the graph structure of
road networks is static, missing opportunities for further traffic flow
optimization. We are interested in optimizing traffic flows as a new type of
graph-based problem, where the graph structure of a road network can adapt to
traffic conditions in real time. In particular, we focus on the dynamic
configuration of traffic-lane directions, which can help balance the usage of
traffic lanes in opposite directions. The rise of connected autonomous vehicles
offers an opportunity to apply this type of dynamic traffic optimization at a
large scale. The existing techniques for optimizing lane-directions are however
not suitable for dynamic traffic environments due to their high computational
complexity and the static nature.
  In this paper, we propose an efficient traffic optimization solution, called
Coordinated Learning-based Lane Allocation (CLLA), which is suitable for
dynamic configuration of lane-directions. CLLA consists of a two-layer
multi-agent architecture, where the bottom-layer agents use a machine learning
technique to find a suitable configuration of lane-directions around individual
road intersections. The lane-direction changes proposed by the learning agents
are then coordinated at a higher level to reduce the negative impact of the
changes on other parts of the road network. Our experimental results show that
CLLA can reduce the average travel time significantly in congested road
networks. We believe our method is general enough to be applied to other types
of networks as well.
