Learning Dense Visual Correspondences in Simulation to Smooth and Fold
  Real Fabrics
  Robotic fabric manipulation is challenging due to the infinite dimensional
configuration space, self-occlusion, and complex dynamics of fabrics. There has
been significant prior work on learning policies for specific deformable
manipulation tasks, but comparatively less focus on algorithms which can
efficiently learn many different tasks. In this paper, we learn visual
correspondences for deformable fabrics across different configurations in
simulation and show that this representation can be used to design policies for
a variety of tasks. Given a single demonstration of a new task from an initial
fabric configuration, the learned correspondences can be used to compute
geometrically equivalent actions in a new fabric configuration. This makes it
possible to robustly imitate a broad set of multi-step fabric smoothing and
folding tasks on multiple physical robotic systems. The resulting policies
achieve 80.3% average task success rate across 10 fabric manipulation tasks on
two different robotic systems, the da Vinci surgical robot and the ABB YuMi.
Results also suggest robustness to fabrics of various colors, sizes, and
shapes. See https://tinyurl.com/fabric-descriptors for supplementary material
and videos.
