Hamiltonian Operator Disentanglement of Content and Motion in Image
  Sequences
  We introduce a deep generative model for image sequences that reliably
factorise the latent space into content and motion variables. To model the
diverse dynamics, we split the motion space into subspaces and introduce a
unique Hamiltonian operator for each subspace. The Hamiltonian formulation
provides reversible dynamics that constrain the evolution of the motion path
along the low-dimensional manifold and conserves learnt invariant properties.
The explicit split of the motion space decomposes the Hamiltonian into symmetry
groups and gives long-term separability of the dynamics. This split also means
we can learn content representations that are easy to interpret and control. We
demonstrate the utility of our model by swapping the motion of two videos,
generating long term sequences of various actions from a given image,
unconditional sequence generation and image rotations.
