Human AI interaction loop training: New approach for interactive
  reinforcement learning
  Reinforcement Learning (RL) in various decision-making tasks of machine
learning provides effective results with an agent learning from a stand-alone
reward function. However, it presents unique challenges with large amounts of
environment states and action spaces, as well as in the determination of
rewards. This complexity, coming from high dimensionality and continuousness of
the environments considered herein, calls for a large number of learning trials
to learn about the environment through Reinforcement Learning. Imitation
Learning (IL) offers a promising solution for those challenges using a teacher.
In IL, the learning process can take advantage of human-sourced assistance
and/or control over the agent and environment. A human teacher and an agent
learner are considered in this study. The teacher takes part in the agent
training towards dealing with the environment, tackling a specific objective,
and achieving a predefined goal. Within that paradigm, however, existing IL
approaches have the drawback of expecting extensive demonstration information
in long-horizon problems. This paper proposes a novel approach combining IL
with different types of RL methods, namely state action reward state action
(SARSA) and asynchronous advantage actor-critic (A3C) agents, to overcome the
problems of both stand-alone systems. It is addressed how to effectively
leverage the teacher feedback, be it direct binary or indirect detailed for the
agent learner to learn sequential decision-making policies. The results of this
study on various OpenAI Gym environments show that this algorithmic method can
be incorporated with different combinations, significantly decreases both human
endeavor and tedious exploration process.
