Learning of feature points without additional supervision improves
  reinforcement learning from images
  In many control problems that include vision, optimal controls can be
inferred from the location of the objects in the scene. This information can be
represented using feature points, which is a list of spatial locations in
learned feature maps of an input image. Previous works show that feature points
learned using unsupervised pre-training or human supervision can provide good
features for control tasks. In this paper, we show that it is possible to learn
efficient feature point representations end-to-end, without the need for
unsupervised pre-training, decoders, or additional losses. Our proposed
architecture consists of a differentiable feature point extractor that feeds
the coordinates of the estimated feature points directly to a soft actor-critic
agent. The proposed algorithm yields performance competitive to the
state-of-the art on DeepMind Control Suite tasks.
