Microscopic Traffic Simulation by Cooperative Multi-agent Deep
  Reinforcement Learning
  Expert human drivers perform actions relying on traffic laws and their
previous experience. While traffic laws are easily embedded into an artificial
brain, modeling human complex behaviors which come from past experience is a
more challenging task. One of these behaviors is the capability of
communicating intentions and negotiating the right of way through driving
actions, as when a driver is entering a crowded roundabout and observes other
cars movements to guess the best time to merge in. In addition, each driver has
its own unique driving style, which is conditioned by both its personal
characteristics, such as age and quality of sight, and external factors, such
as being late or in a bad mood. For these reasons, the interaction between
different drivers is not trivial to simulate in a realistic manner. In this
paper, this problem is addressed by developing a microscopic simulator using a
Deep Reinforcement Learning Algorithm based on a combination of visual frames,
representing the perception around the vehicle, and a vector of numerical
parameters. In particular, the algorithm called Asynchronous Advantage
Actor-Critic has been extended to a multi-agent scenario in which every agent
needs to learn to interact with other similar agents. Moreover, the model
includes a novel architecture such that the driving style of each vehicle is
adjustable by tuning some of its input parameters, permitting to simulate
drivers with different levels of aggressiveness and desired cruising speeds.
