MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale
  General-purpose robotic systems must master a large repertoire of diverse
skills to be useful in a range of daily tasks. While reinforcement learning
provides a powerful framework for acquiring individual behaviors, the time
needed to acquire each skill makes the prospect of a generalist robot trained
with RL daunting. In this paper, we study how a large-scale collective robotic
learning system can acquire a repertoire of behaviors simultaneously, sharing
exploration, experience, and representations across tasks. In this framework
new tasks can be continuously instantiated from previously learned tasks
improving overall performance and capabilities of the system. To instantiate
this system, we develop a scalable and intuitive framework for specifying new
tasks through user-provided examples of desired outcomes, devise a multi-robot
collective learning system for data collection that simultaneously collects
experience for multiple tasks, and develop a scalable and generalizable
multi-task deep reinforcement learning method, which we call MT-Opt. We
demonstrate how MT-Opt can learn a wide range of skills, including semantic
picking (i.e., picking an object from a particular category), placing into
various fixtures (e.g., placing a food item onto a plate), covering, aligning,
and rearranging. We train and evaluate our system on a set of 12 real-world
tasks with data collected from 7 robots, and demonstrate the performance of our
system both in terms of its ability to generalize to structurally similar new
tasks, and acquire distinct new tasks more quickly by leveraging past
experience. We recommend viewing the videos at
https://karolhausman.github.io/mt-opt/
