When Do Drivers Concentrate? Attention-based Driver Behavior Modeling
  With Deep Reinforcement Learning
  Driver distraction a significant risk to driving safety. Apart from spatial
domain, research on temporal inattention is also necessary. This paper aims to
figure out the pattern of drivers' temporal attention allocation. In this
paper, we propose an actor-critic method - Attention-based Twin Delayed Deep
Deterministic policy gradient (ATD3) algorithm to approximate a driver' s
action according to observations and measure the driver' s attention allocation
for consecutive time steps in car-following model. Considering reaction time,
we construct the attention mechanism in the actor network to capture temporal
dependencies of consecutive observations. In the critic network, we employ Twin
Delayed Deep Deterministic policy gradient algorithm (TD3) to address
overestimated value estimates persisting in the actor-critic algorithm. We
conduct experiments on real-world vehicle trajectory datasets and show that the
accuracy of our proposed approach outperforms seven baseline algorithms.
Moreover, the results reveal that the attention of the drivers in smooth
vehicles is uniformly distributed in previous observations while they keep
their attention to recent observations when sudden decreases of relative speeds
occur. This study is the first contribution to drivers' temporal attention and
provides scientific support for safety measures in transportation systems from
the perspective of data mining.
