Error-Aware Imitation Learning from Teleoperation Data for Mobile
  Manipulation
  In mobile manipulation (MM), robots can both navigate within and interact
with their environment and are thus able to complete many more tasks than
robots only capable of navigation or manipulation. In this work, we explore how
to apply imitation learning (IL) to learn continuous visuo-motor policies for
MM tasks. Much prior work has shown that IL can train visuo-motor policies for
either manipulation or navigation domains, but few works have applied IL to the
MM domain. Doing this is challenging for two reasons: on the data side, current
interfaces make collecting high-quality human demonstrations difficult, and on
the learning side, policies trained on limited data can suffer from covariate
shift when deployed. To address these problems, we first propose Mobile
Manipulation RoboTurk (MoMaRT), a novel teleoperation framework allowing
simultaneous navigation and manipulation of mobile manipulators, and collect a
first-of-its-kind large scale dataset in a realistic simulated kitchen setting.
We then propose a learned error detection system to address the covariate shift
by detecting when an agent is in a potential failure state. We train performant
IL policies and error detectors from this data, and achieve over 45% task
success rate and 85% error detection success rate across multiple multi-stage
tasks when trained on expert data. Codebase, datasets, visualization, and more
available at https://sites.google.com/view/il-for-mm/home.
