Accelerating Learning in Constructive Predictive Frameworks with the
  Successor Representation
  Here we propose using the successor representation (SR) to accelerate
learning in a constructive knowledge system based on general value functions
(GVFs). In real-world settings like robotics for unstructured and dynamic
environments, it is infeasible to model all meaningful aspects of a system and
its environment by hand due to both complexity and size. Instead, robots must
be capable of learning and adapting to changes in their environment and task,
incrementally constructing models from their own experience. GVFs, taken from
the field of reinforcement learning (RL), are a way of modeling the world as
predictive questions. One approach to such models proposes a massive network of
interconnected and interdependent GVFs, which are incrementally added over
time. It is reasonable to expect that new, incrementally added predictions can
be learned more swiftly if the learning process leverages knowledge gained from
past experience. The SR provides such a means of separating the dynamics of the
world from the prediction targets and thus capturing regularities that can be
reused across multiple GVFs. As a primary contribution of this work, we show
that using SR-based predictions can improve sample efficiency and learning
speed in a continual learning setting where new predictions are incrementally
added and learned over time. We analyze our approach in a grid-world and then
demonstrate its potential on data from a physical robot arm.
