Combining Reinforcement Learning with Model Predictive Control for
  On-Ramp Merging
  We consider the problem of designing an algorithm to allow a car to
autonomously merge on to a highway from an on-ramp. Two broad classes of
techniques have been proposed to solve motion planning problems in autonomous
driving: Model Predictive Control (MPC) and Reinforcement Learning (RL). In
this paper, we first establish the strengths and weaknesses of state-of-the-art
MPC and RL-based techniques through simulations. We show that the performance
of the RL agent is worse than that of the MPC solution from the perspective of
safety and robustness to out-of-distribution traffic patterns, i.e., traffic
patterns which were not seen by the RL agent during training. On the other
hand, the performance of the RL agent is better than that of the MPC solution
when it comes to efficiency and passenger comfort. We subsequently present an
algorithm which blends the model-free RL agent with the MPC solution and show
that it provides better trade-offs between all metrics -- passenger comfort,
efficiency, crash rate and robustness.
