Pretraining Representations for Data-Efficient Reinforcement Learning
  Data efficiency is a key challenge for deep reinforcement learning. We
address this problem by using unlabeled data to pretrain an encoder which is
then finetuned on a small amount of task-specific data. To encourage learning
representations which capture diverse aspects of the underlying MDP, we employ
a combination of latent dynamics modelling and unsupervised goal-conditioned
RL. When limited to 100k steps of interaction on Atari games (equivalent to two
hours of human experience), our approach significantly surpasses prior work
combining offline representation pretraining with task-specific finetuning, and
compares favourably with other pretraining methods that require orders of
magnitude more data. Our approach shows particular promise when combined with
larger models as well as more diverse, task-aligned observational data --
approaching human-level performance and data-efficiency on Atari in our best
setting. We provide code associated with this work at
https://github.com/mila-iqia/SGI.
