Spatial Influence-aware Reinforcement Learning for Intelligent
  Transportation System
  Intelligent transportation systems (ITSs) are envisioned to be crucial for
smart cities, which aims at improving traffic flow to improve the life quality
of urban residents and reducing congestion to improve the efficiency of
commuting. However, several challenges need to be resolved before such systems
can be deployed, for example, conventional solutions for Markov decision
process (MDP) and single-agent Reinforcement Learning (RL) algorithms suffer
from poor scalability, and multi-agent systems suffer from poor communication
and coordination. In this paper, we explore the potential of mutual information
sharing, or in other words, spatial influence based communication, to optimize
traffic light control policy. First, we mathematically analyze the
transportation system. We conclude that the transportation system does not have
stationary Nash Equilibrium, thereby reinforcement learning algorithms offer
suitable solutions. Secondly, we describe how to build a multi-agent Deep
Deterministic Policy Gradient (DDPG) system with spatial influence and social
group utility incorporated. Then we utilize the grid topology road network to
empirically demonstrate the scalability of the new system. We demonstrate three
types of directed communications to show the effect of directions of social
influence on the entire network utility and individual utility. Lastly, we
define "selfish index" and analyze the effect of it on total group utility.
