IRIS: Implicit Reinforcement without Interaction at Scale for Learning
  Control from Offline Robot Manipulation Data
  Learning from offline task demonstrations is a problem of great interest in
robotics. For simple short-horizon manipulation tasks with modest variation in
task instances, offline learning from a small set of demonstrations can produce
controllers that successfully solve the task. However, leveraging a fixed batch
of data can be problematic for larger datasets and longer-horizon tasks with
greater variations. The data can exhibit substantial diversity and consist of
suboptimal solution approaches. In this paper, we propose Implicit
Reinforcement without Interaction at Scale (IRIS), a novel framework for
learning from large-scale demonstration datasets. IRIS factorizes the control
problem into a goal-conditioned low-level controller that imitates short
demonstration sequences and a high-level goal selection mechanism that sets
goals for the low-level and selectively combines parts of suboptimal solutions
leading to more successful task completions. We evaluate IRIS across three
datasets, including the RoboTurk Cans dataset collected by humans via
crowdsourcing, and show that performant policies can be learned from purely
offline learning. Additional results at
https://sites.google.com/stanford.edu/iris/ .
