Visual Grounding of Learned Physical Models
  Humans intuitively recognize objects' physical properties and predict their
motion, even when the objects are engaged in complicated interactions. The
abilities to perform physical reasoning and to adapt to new environments, while
intrinsic to humans, remain challenging to state-of-the-art computational
models. In this work, we present a neural model that simultaneously reasons
about physics and makes future predictions based on visual and dynamics priors.
The visual prior predicts a particle-based representation of the system from
visual observations. An inference module operates on those particles,
predicting and refining estimates of particle locations, object states, and
physical parameters, subject to the constraints imposed by the dynamics prior,
which we refer to as visual grounding. We demonstrate the effectiveness of our
method in environments involving rigid objects, deformable materials, and
fluids. Experiments show that our model can infer the physical properties
within a few observations, which allows the model to quickly adapt to unseen
scenarios and make accurate predictions into the future.
