Combining Benefits from Trajectory Optimization and Deep Reinforcement
  Learning
  Recent breakthroughs both in reinforcement learning and trajectory
optimization have made significant advances towards real world robotic system
deployment. Reinforcement learning (RL) can be applied to many problems without
needing any modeling or intuition about the system, at the cost of high sample
complexity and the inability to prove any metrics about the learned policies.
Trajectory optimization (TO) on the other hand allows for stability and
robustness analyses on generated motions and trajectories, but is only as good
as the often over-simplified derived model, and may have prohibitively
expensive computation times for real-time control. This paper seeks to combine
the benefits from these two areas while mitigating their drawbacks by (1)
decreasing RL sample complexity by using existing knowledge of the problem with
optimal control, and (2) providing an upper bound estimate on the
time-to-arrival of the combined learned-optimized policy, allowing online
policy deployment at any point in the training process by using the TO as a
worst-case scenario action. This method is evaluated for a car model, with
applicability to any mobile robotic system. A video showing policy execution
comparisons can be found at https://youtu.be/mv2xw83NyWU .
