NewtonianVAE: Proportional Control and Goal Identification from Pixels
  via Physical Latent Spaces
  Learning low-dimensional latent state space dynamics models has been a
powerful paradigm for enabling vision-based planning and learning for control.
We introduce a latent dynamics learning framework that is uniquely designed to
induce proportional controlability in the latent space, thus enabling the use
of much simpler controllers than prior work. We show that our learned dynamics
model enables proportional control from pixels, dramatically simplifies and
accelerates behavioural cloning of vision-based controllers, and provides
interpretable goal discovery when applied to imitation learning of switching
controllers from demonstration.
