Safe Model-based Off-policy Reinforcement Learning for Eco-Driving in
  Connected and Automated Hybrid Electric Vehicles
  Connected and Automated Hybrid Electric Vehicles have the potential to reduce
fuel consumption and travel time in real-world driving conditions. The
eco-driving problem seeks to design optimal speed and power usage profiles
based upon look-ahead information from connectivity and advanced mapping
features. Recently, Deep Reinforcement Learning (DRL) has been applied to the
eco-driving problem. While the previous studies synthesize simulators and
model-free DRL to reduce online computation, this work proposes a Safe
Off-policy Model-Based Reinforcement Learning algorithm for the eco-driving
problem. The advantages over the existing literature are three-fold. First, the
combination of off-policy learning and the use of a physics-based model
improves the sample efficiency. Second, the training does not require any
extrinsic rewarding mechanism for constraint satisfaction. Third, the
feasibility of trajectory is guaranteed by using a safe set approximated by
deep generative models.
  The performance of the proposed method is benchmarked against a baseline
controller representing human drivers, a previously designed model-free DRL
strategy, and the wait-and-see optimal solution. In simulation, the proposed
algorithm leads to a policy with a higher average speed and a better fuel
economy compared to the model-free agent. Compared to the baseline controller,
the learned strategy reduces the fuel consumption by more than 21\% while
keeping the average speed comparable.
