SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred
  from Vision
  A recently proposed class of models attempts to learn latent dynamics from
high-dimensional observations, like images, using priors informed by
Hamiltonian mechanics. While these models have important potential applications
in areas like robotics or autonomous driving, there is currently no good way to
evaluate their performance: existing methods primarily rely on image
reconstruction quality, which does not always reflect the quality of the learnt
latent dynamics. In this work, we empirically highlight the problems with the
existing measures and develop a set of new measures, including a binary
indicator of whether the underlying Hamiltonian dynamics have been faithfully
captured, which we call Symplecticity Metric or SyMetric. Our measures take
advantage of the known properties of Hamiltonian dynamics and are more
discriminative of the model's ability to capture the underlying dynamics than
reconstruction error. Using SyMetric, we identify a set of architectural
choices that significantly improve the performance of a previously proposed
model for inferring latent dynamics from pixels, the Hamiltonian Generative
Network (HGN). Unlike the original HGN, the new HGN++ is able to discover an
interpretable phase space with physically meaningful latents on some datasets.
Furthermore, it is stable for significantly longer rollouts on a diverse range
of 13 datasets, producing rollouts of essentially infinite length both forward
and backwards in time with no degradation in quality on a subset of the
datasets.
