Offline Reinforcement Learning for Road Traffic Control
  Traffic signal control is an important problem in urban mobility with a
significant potential of economic and environmental impact. While there is a
growing interest in Reinforcement Learning (RL) for traffic control, the work
so far has focussed on learning through interactions which, in practice, is
costly. Instead, real experience data on traffic is available and could be
exploited at minimal costs. Recent progress in offline or batch RL has enabled
just that. Model-based offline RL methods, in particular, have been shown to
generalize to the experience data much better than others. We build a
model-based learning framework, A-DAC, which infers a Markov Decision Process
(MDP) from dataset with pessimistic costs built in to deal with data
uncertainties. The costs are modeled through an adaptive shaping of rewards in
the MDP which provides better regularization of data compared to the prior
related work. A-DAC is evaluated on a complex signalized roundabout using
multiple datasets varying in size and in batch collection policy. The
evaluation results show that it is possible to build high performance control
policies in a data efficient manner using simplistic batch collection policies.
