Mutual Information Based Method for Unsupervised Disentanglement of
  Video Representation
  Video Prediction is an interesting and challenging task of predicting future
frames from a given set context frames that belong to a video sequence. Video
prediction models have found prospective applications in Maneuver Planning,
Health care, Autonomous Navigation and Simulation. One of the major challenges
in future frame generation is due to the high dimensional nature of visual
data. In this work, we propose Mutual Information Predictive Auto-Encoder
(MIPAE) framework, that reduces the task of predicting high dimensional video
frames by factorising video representations into content and low dimensional
pose latent variables that are easy to predict. A standard LSTM network is used
to predict these low dimensional pose representations. Content and the
predicted pose representations are decoded to generate future frames. Our
approach leverages the temporal structure of the latent generative factors of a
video and a novel mutual information loss to learn disentangled video
representations. We also propose a metric based on mutual information gap (MIG)
to quantitatively access the effectiveness of disentanglement on DSprites and
MPI3D-real datasets. MIG scores corroborate with the visual superiority of
frames predicted by MIPAE. We also compare our method quantitatively on
evaluation metrics LPIPS, SSIM and PSNR.
