A Conceptual Framework for Externally-influenced Agents: An Assisted
  Reinforcement Learning Review
  A long-term goal of reinforcement learning agents is to be able to perform
tasks in complex real-world scenarios. The use of external information is one
way of scaling agents to more complex problems. However, there is a general
lack of collaboration or interoperability between different approaches using
external information. In this work, while reviewing externally-influenced
methods, we propose a conceptual framework and taxonomy for assisted
reinforcement learning, aimed at fostering collaboration by classifying and
comparing various methods that use external information in the learning
process. The proposed taxonomy details the relationship between the external
information source and the learner agent, highlighting the process of
information decomposition, structure, retention, and how it can be used to
influence agent learning. As well as reviewing state-of-the-art methods, we
identify current streams of reinforcement learning that use external
information in order to improve the agent's performance and its decision-making
process. These include heuristic reinforcement learning, interactive
reinforcement learning, learning from demonstration, transfer learning, and
learning from multiple sources, among others. These streams of reinforcement
learning operate with the shared objective of scaffolding the learner agent.
Lastly, we discuss further possibilities for future work in the field of
assisted reinforcement learning systems.
