DiffSkill: Skill Abstraction from Differentiable Physics for Deformable
  Object Manipulations with Tools
  We consider the problem of sequential robotic manipulation of deformable
objects using tools. Previous works have shown that differentiable physics
simulators provide gradients to the environment state and help trajectory
optimization to converge orders of magnitude faster than model-free
reinforcement learning algorithms for deformable object manipulation. However,
such gradient-based trajectory optimization typically requires access to the
full simulator states and can only solve short-horizon, single-skill tasks due
to local optima. In this work, we propose a novel framework, named DiffSkill,
that uses a differentiable physics simulator for skill abstraction to solve
long-horizon deformable object manipulation tasks from sensory observations. In
particular, we first obtain short-horizon skills using individual tools from a
gradient-based optimizer, using the full state information in a differentiable
simulator; we then learn a neural skill abstractor from the demonstration
trajectories which takes RGBD images as input. Finally, we plan over the skills
by finding the intermediate goals and then solve long-horizon tasks. We show
the advantages of our method in a new set of sequential deformable object
manipulation tasks compared to previous reinforcement learning algorithms and
compared to the trajectory optimizer.
