Tensor-based Cooperative Control for Large Scale Multi-intersection
  Traffic Signal Using Deep Reinforcement Learning and Imitation Learning
  Traffic signal control has long been considered as a critical topic in
intelligent transportation systems. Most existing learning methods mainly focus
on isolated intersections and suffer from inefficient training. This paper aims
at the cooperative control for large scale multi-intersection traffic signal,
in which a novel end-to-end learning based model is established and the
efficient training method is proposed correspondingly. In the proposed model,
the input traffic status in multi-intersections is represented by a tensor,
which not only significantly reduces dimensionality than using a single matrix
but also avoids information loss. For the output, a multidimensional boolean
vector is employed for the control policy to indicate whether the signal state
changes or not, which simplifies the representation and abides the practical
phase changing rules. In the proposed model, a multi-task learning structure is
used to get the cooperative policy by learning. Instead of only using the
reinforcement learning to train the model, we employ imitation learning to
integrate a rule based model with neural networks to do the pre-training, which
provides a reliable and satisfactory stage solution and greatly accelerates the
convergence. Afterwards, the reinforcement learning method is adopted to
continue the fine training, where proximal policy optimization algorithm is
incorporated to solve the policy collapse problem in multi-dimensional output
situation. In numerical experiments, the advantages of the proposed model are
demonstrated with comparison to the related state-of-the-art methods.
