Learning Gaussian Policies from Corrective Human Feedback
  Learning from human feedback is a viable alternative to control design that
does not require modelling or control expertise. Particularly, learning from
corrective advice garners advantages over evaluative feedback as it is a more
intuitive and scalable format. The current state-of-the-art in this field,
COACH, has proven to be a effective approach for confined problems. However, it
parameterizes the policy with Radial Basis Function networks, which require
meticulous feature space engineering for higher order systems. We introduce
Gaussian Process Coach (GPC), where feature space engineering is avoided by
employing Gaussian Processes. In addition, we use the available policy
uncertainty to 1) inquire feedback samples of maximal utility and 2) to adapt
the learning rate to the teacher's learning phase. We demonstrate that the
novel algorithm outperforms the current state-of-the-art in final performance,
convergence rate and robustness to erroneous feedback in OpenAI Gym continuous
control benchmarks, both for simulated and real human teachers.
