Goal-Aware Prediction: Learning to Model What Matters
  Learned dynamics models combined with both planning and policy learning
algorithms have shown promise in enabling artificial agents to learn to perform
many diverse tasks with limited supervision. However, one of the fundamental
challenges in using a learned forward dynamics model is the mismatch between
the objective of the learned model (future state reconstruction), and that of
the downstream planner or policy (completing a specified task). This issue is
exacerbated by vision-based control tasks in diverse real-world environments,
where the complexity of the real world dwarfs model capacity. In this paper, we
propose to direct prediction towards task relevant information, enabling the
model to be aware of the current task and encouraging it to only model relevant
quantities of the state space, resulting in a learning objective that more
closely matches the downstream task. Further, we do so in an entirely
self-supervised manner, without the need for a reward function or image labels.
We find that our method more effectively models the relevant parts of the scene
conditioned on the goal, and as a result outperforms standard task-agnostic
dynamics models and model-free reinforcement learning.
