SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object
  Manipulation
  Manipulating deformable objects has long been a challenge in robotics due to
its high dimensional state representation and complex dynamics. Recent success
in deep reinforcement learning provides a promising direction for learning to
manipulate deformable objects with data driven methods. However, existing
reinforcement learning benchmarks only cover tasks with direct state
observability and simple low-dimensional dynamics or with relatively simple
image-based environments, such as those with rigid objects. In this paper, we
present SoftGym, a set of open-source simulated benchmarks for manipulating
deformable objects, with a standard OpenAI Gym API and a Python interface for
creating new environments. Our benchmark will enable reproducible research in
this important area. Further, we evaluate a variety of algorithms on these
tasks and highlight challenges for reinforcement learning algorithms, including
dealing with a state representation that has a high intrinsic dimensionality
and is partially observable. The experiments and analysis indicate the
strengths and limitations of existing methods in the context of deformable
object manipulation that can help point the way forward for future methods
development. Code and videos of the learned policies can be found on our
project website.
