Assessment of Reward Functions in Reinforcement Learning for Multi-Modal
  Urban Traffic Control under Real-World limitations
  Reinforcement Learning is proving a successful tool that can manage urban
intersections with a fraction of the effort required to curate traditional
traffic controllers. However, literature on the introduction and control of
pedestrians to such intersections is scarce. Furthermore, it is unclear what
traffic state variables should be used as reward to obtain the best agent
performance. This paper robustly evaluates 30 different Reinforcement Learning
reward functions for controlling intersections serving pedestrians and vehicles
covering the main traffic state variables available via modern vision-based
sensors. Some rewards proposed in previous literature solely for vehicular
traffic are extended to pedestrians while new ones are introduced. We use a
calibrated model in terms of demand, sensors, green times and other operational
constraints of a real intersection in Greater Manchester, UK. The assessed
rewards can be classified in 5 groups depending on the magnitudes used: queues,
waiting time, delay, average speed and throughput in the junction. The
performance of different agents, in terms of waiting time, is compared across
different demand levels, from normal operation to saturation of traditional
adaptive controllers. We find that those rewards maximising the speed of the
network obtain the lowest waiting time for vehicles and pedestrians
simultaneously, closely followed by queue minimisation, demonstrating better
performance than other previously proposed methods.
