From Language to Goals: Inverse Reinforcement Learning for Vision-Based
  Instruction Following
  Reinforcement learning is a promising framework for solving control problems,
but its use in practical situations is hampered by the fact that reward
functions are often difficult to engineer. Specifying goals and tasks for
autonomous machines, such as robots, is a significant challenge:
conventionally, reward functions and goal states have been used to communicate
objectives. But people can communicate objectives to each other simply by
describing or demonstrating them. How can we build learning algorithms that
will allow us to tell machines what we want them to do? In this work, we
investigate the problem of grounding language commands as reward functions
using inverse reinforcement learning, and argue that language-conditioned
rewards are more transferable than language-conditioned policies to new
environments. We propose language-conditioned reward learning (LC-RL), which
grounds language commands as a reward function represented by a deep neural
network. We demonstrate that our model learns rewards that transfer to novel
tasks and environments on realistic, high-dimensional visual environments with
natural language commands, whereas directly learning a language-conditioned
policy leads to poor performance.
