KINet: Keypoint Interaction Networks for Unsupervised Forward Modeling
  Object-centric representation is an essential abstraction for physical
reasoning and forward prediction. Most existing approaches learn this
representation through extensive supervision (e.g., object class and bounding
box) although such ground-truth information is not readily accessible in
reality. To address this, we introduce KINet (Keypoint Interaction Network) --
an end-to-end unsupervised framework to reason about object interactions in
complex systems based on a keypoint representation. Using visual observations,
our model learns to associate objects with keypoint coordinates and discovers a
graph representation of the system as a set of keypoint embeddings and their
relations. It then learns an action-conditioned forward model using contrastive
estimation to predict future keypoint states. By learning to perform physical
reasoning in the keypoint space, our model automatically generalizes to
scenarios with a different number of objects, and novel object geometries.
Experiments demonstrate the effectiveness of our model to accurately perform
forward prediction and learn plannable object-centric representations which can
also be used in downstream model-based control tasks.
