Learning to Act by Predicting the Future
  We present an approach to sensorimotor control in immersive environments. Our
approach utilizes a high-dimensional sensory stream and a lower-dimensional
measurement stream. The cotemporal structure of these streams provides a rich
supervisory signal, which enables training a sensorimotor control model by
interacting with the environment. The model is trained using supervised
learning techniques, but without extraneous supervision. It learns to act based
on raw sensory input from a complex three-dimensional environment. The
presented formulation enables learning without a fixed goal at training time,
and pursuing dynamically changing goals at test time. We conduct extensive
experiments in three-dimensional simulations based on the classical
first-person game Doom. The results demonstrate that the presented approach
outperforms sophisticated prior formulations, particularly on challenging
tasks. The results also show that trained models successfully generalize across
environments and goals. A model trained using the presented approach won the
Full Deathmatch track of the Visual Doom AI Competition, which was held in
previously unseen environments.
