Unsupervised Control Through Non-Parametric Discriminative Rewards
  Learning to control an environment without hand-crafted rewards or expert
data remains challenging and is at the frontier of reinforcement learning
research. We present an unsupervised learning algorithm to train agents to
achieve perceptually-specified goals using only a stream of observations and
actions. Our agent simultaneously learns a goal-conditioned policy and a goal
achievement reward function that measures how similar a state is to the goal
state. This dual optimization leads to a co-operative game, giving rise to a
learned reward function that reflects similarity in controllable aspects of the
environment instead of distance in the space of observations. We demonstrate
the efficacy of our agent to learn, in an unsupervised manner, to reach a
diverse set of goals on three domains -- Atari, the DeepMind Control Suite and
DeepMind Lab.
