Deep Reinforcement Learning with Embedded LQR Controllers
  Reinforcement learning is a model-free optimal control method that optimizes
a control policy through direct interaction with the environment. For reaching
tasks that end in regulation, popular discrete-action methods are not well
suited due to chattering in the goal state. We compare three different ways to
solve this problem through combining reinforcement learning with classical LQR
control. In particular, we introduce a method that integrates LQR control into
the action set, allowing generalization and avoiding fixing the computed
control in the replay memory if it is based on learned dynamics. We also embed
LQR control into a continuous-action method. In all cases, we show that adding
LQR control can improve performance, although the effect is more profound if it
can be used to augment a discrete action set.
