Efficient Bimanual Manipulation Using Learned Task Schemas
  We address the problem of effectively composing skills to solve sparse-reward
tasks in the real world. Given a set of parameterized skills (such as exerting
a force or doing a top grasp at a location), our goal is to learn policies that
invoke these skills to efficiently solve such tasks. Our insight is that for
many tasks, the learning process can be decomposed into learning a
state-independent task schema (a sequence of skills to execute) and a policy to
choose the parameterizations of the skills in a state-dependent manner. For
such tasks, we show that explicitly modeling the schema's state-independence
can yield significant improvements in sample efficiency for model-free
reinforcement learning algorithms. Furthermore, these schemas can be
transferred to solve related tasks, by simply re-learning the parameterizations
with which the skills are invoked. We find that doing so enables learning to
solve sparse-reward tasks on real-world robotic systems very efficiently. We
validate our approach experimentally over a suite of robotic bimanual
manipulation tasks, both in simulation and on real hardware. See videos at
http://tinyurl.com/chitnis-schema.
