Video Reenactment as Inductive Bias for Content-Motion Disentanglement
  Independent components within low-dimensional representations are essential
inputs in several downstream tasks, and provide explanations over the observed
data. Video-based disentangled factors of variation provide low-dimensional
representations that can be identified and used to feed task-specific models.
We introduce MTC-VAE, a self-supervised motion-transfer VAE model to
disentangle motion and content from videos. Unlike previous work on video
content-motion disentanglement, we adopt a chunk-wise modeling approach and
take advantage of the motion information contained in spatiotemporal
neighborhoods. Our model yields independent per-chunk representations that
preserve temporal consistency. Hence, we reconstruct whole videos in a single
forward-pass. We extend the ELBO's log-likelihood term and include a Blind
Reenactment Loss as an inductive bias to leverage motion disentanglement, under
the assumption that swapping motion features yields reenactment between two
videos. We evaluate our model with recently-proposed disentanglement metrics
and show that it outperforms a variety of methods for video motion-content
disentanglement. Experiments on video reenactment show the effectiveness of our
disentanglement in the input space where our model outperforms the baselines in
reconstruction quality and motion alignment.
