Information is Power: Intrinsic Control via Information Capture
  Humans and animals explore their environment and acquire useful skills even
in the absence of clear goals, exhibiting intrinsic motivation. The study of
intrinsic motivation in artificial agents is concerned with the following
question: what is a good general-purpose objective for an agent? We study this
question in dynamic partially-observed environments, and argue that a compact
and general learning objective is to minimize the entropy of the agent's state
visitation estimated using a latent state-space model. This objective induces
an agent to both gather information about its environment, corresponding to
reducing uncertainty, and to gain control over its environment, corresponding
to reducing the unpredictability of future world states. We instantiate this
approach as a deep reinforcement learning agent equipped with a deep
variational Bayes filter. We find that our agent learns to discover, represent,
and exercise control of dynamic objects in a variety of partially-observed
environments sensed with visual observations without extrinsic reward.
