Training Agents using Upside-Down Reinforcement Learning
  We develop Upside-Down Reinforcement Learning (UDRL), a method for learning
to act using only supervised learning techniques. Unlike traditional
algorithms, UDRL does not use reward prediction or search for an optimal
policy. Instead, it trains agents to follow commands such as "obtain so much
total reward in so much time." Many of its general principles are outlined in a
companion report; the goal of this paper is to develop a practical learning
algorithm and show that this conceptually simple perspective on agent training
can produce a range of rewarding behaviors for multiple episodic environments.
Experiments show that on some tasks UDRL's performance can be surprisingly
competitive with, and even exceed that of some traditional baseline algorithms
developed over decades of research. Based on these results, we suggest that
alternative approaches to expected reward maximization have an important role
to play in training useful autonomous agents.
