Human-like Energy Management Based on Deep Reinforcement Learning and
  Historical Driving Experiences
  Development of hybrid electric vehicles depends on an advanced and efficient
energy management strategy (EMS). With online and real-time requirements in
mind, this article presents a human-like energy management framework for hybrid
electric vehicles according to deep reinforcement learning methods and
collected historical driving data. The hybrid powertrain studied has a
series-parallel topology, and its control-oriented modeling is founded first.
Then, the distinctive deep reinforcement learning (DRL) algorithm, named deep
deterministic policy gradient (DDPG), is introduced. To enhance the derived
power split controls in the DRL framework, the global optimal control
trajectories obtained from dynamic programming (DP) are regarded as expert
knowledge to train the DDPG model. This operation guarantees the optimality of
the proposed control architecture. Moreover, the collected historical driving
data based on experienced drivers are employed to replace the DP-based
controls, and thus construct the human-like EMSs. Finally, different categories
of experiments are executed to estimate the optimality and adaptability of the
proposed human-like EMS. Improvements in fuel economy and convergence rate
indicate the effectiveness of the constructed control structure.
