Transfer with Model Features in Reinforcement Learning
  A key question in Reinforcement Learning is which representation an agent can
learn to efficiently reuse knowledge between different tasks. Recently the
Successor Representation was shown to have empirical benefits for transferring
knowledge between tasks with shared transition dynamics. This paper presents
Model Features: a feature representation that clusters behaviourally equivalent
states and that is equivalent to a Model-Reduction. Further, we present a
Successor Feature model which shows that learning Successor Features is
equivalent to learning a Model-Reduction. A novel optimization objective is
developed and we provide bounds showing that minimizing this objective results
in an increasingly improved approximation of a Model-Reduction. Further, we
provide transfer experiments on randomly generated MDPs which vary in their
transition and reward functions but approximately preserve behavioural
equivalence between states. These results demonstrate that Model Features are
suitable for transfer between tasks with varying transition and reward
functions.
