Finding Useful Predictions by Meta-gradient Descent to Improve
  Decision-making
  In computational reinforcement learning, a growing body of work seeks to
express an agent's model of the world through predictions about future
sensations. In this manuscript we focus on predictions expressed as General
Value Functions: temporally extended estimates of the accumulation of a future
signal. One challenge is determining from the infinitely many predictions that
the agent could possibly make which might support decision-making. In this
work, we contribute a meta-gradient descent method by which an agent can
directly specify what predictions it learns, independent of designer
instruction. To that end, we introduce a partially observable domain suited to
this investigation. We then demonstrate that through interaction with the
environment an agent can independently select predictions that resolve the
partial-observability, resulting in performance similar to expertly chosen
value functions. By learning, rather than manually specifying these
predictions, we enable the agent to identify useful predictions in a
self-supervised manner, taking a step towards truly autonomous systems.
