Unsupervised Object Keypoint Learning using Local Spatial Predictability
  We propose PermaKey, a novel approach to representation learning based on
object keypoints. It leverages the predictability of local image regions from
spatial neighborhoods to identify salient regions that correspond to object
parts, which are then converted to keypoints. Unlike prior approaches, it
utilizes predictability to discover object keypoints, an intrinsic property of
objects. This ensures that it does not overly bias keypoints to focus on
characteristics that are not unique to objects, such as movement, shape, colour
etc. We demonstrate the efficacy of PermaKey on Atari where it learns keypoints
corresponding to the most salient object parts and is robust to certain visual
distractors. Further, on downstream RL tasks in the Atari domain we demonstrate
how agents equipped with our keypoints outperform those using competing
alternatives, even on challenging environments with moving backgrounds or
distractor objects.
