Hybrid Car-Following Strategy based on Deep Deterministic Policy
  Gradient and Cooperative Adaptive Cruise Control
  Deep deterministic policy gradient (DDPG)-based car-following strategy can
break through the constraints of the differential equation model due to the
ability of exploration on complex environments. However, the car-following
performance of DDPG is usually degraded by unreasonable reward function design,
insufficient training, and low sampling efficiency. In order to solve this kind
of problem, a hybrid car-following strategy based on DDPG and cooperative
adaptive cruise control (CACC) is proposed. First, the car-following process is
modeled as the Markov decision process to calculate CACC and DDPG
simultaneously at each frame. Given a current state, two actions are obtained
from CACC and DDPG, respectively. Then, an optimal action, corresponding to the
one offering a larger reward, is chosen as the output of the hybrid strategy.
Meanwhile, a rule is designed to ensure that the change rate of acceleration is
smaller than the desired value. Therefore, the proposed strategy not only
guarantees the basic performance of car-following through CACC but also makes
full use of the advantages of exploration on complex environments via DDPG.
Finally, simulation results show that the car-following performance of the
proposed strategy is improved compared with that of DDPG and CACC.
