Continuous Control for High-Dimensional State Spaces: An Interactive
  Learning Approach
  Deep Reinforcement Learning (DRL) has become a powerful methodology to solve
complex decision-making problems. However, DRL has several limitations when
used in real-world problems (e.g., robotics applications). For instance, long
training times are required and cannot be accelerated in contrast to simulated
environments, and reward functions may be hard to specify/model and/or to
compute. Moreover, the transfer of policies learned in a simulator to the
real-world has limitations (reality gap). On the other hand, machine learning
methods that rely on the transfer of human knowledge to an agent have shown to
be time efficient for obtaining well performing policies and do not require a
reward function. In this context, we analyze the use of human corrective
feedback during task execution to learn policies with high-dimensional state
spaces, by using the D-COACH framework, and we propose new variants of this
framework. D-COACH is a Deep Learning based extension of COACH (COrrective
Advice Communicated by Humans), where humans are able to shape policies through
corrective advice. The enhanced version of D-COACH, which is proposed in this
paper, largely reduces the time and effort of a human for training a policy.
Experimental results validate the efficiency of the D-COACH framework in three
different problems (simulated and with real robots), and show that its enhanced
version reduces the human training effort considerably, and makes it feasible
to learn policies within periods of time in which a DRL agent do not reach any
improvement.
