A Bayesian Treatment of Real-to-Sim for Deformable Object Manipulation
  Deformable object manipulation remains a challenging task in robotics
research. Conventional techniques for parameter inference and state estimation
typically rely on a precise definition of the state space and its dynamics.
While this is appropriate for rigid objects and robot states, it is challenging
to define the state space of a deformable object and how it evolves in time. In
this work, we pose the problem of inferring physical parameters of deformable
objects as a probabilistic inference task defined with a simulator. We propose
a novel methodology for extracting state information from image sequences via a
technique to represent the state of a deformable object as a distribution
embedding. This allows to incorporate noisy state observations directly into
modern Bayesian simulation-based inference tools in a principled manner. Our
experiments confirm that we can estimate posterior distributions of physical
properties, such as elasticity, friction and scale of highly deformable
objects, such as cloth and ropes. Overall, our method addresses the real-to-sim
problem probabilistically and helps to better represent the evolution of the
state of deformable objects.
