Disentangling Space and Time in Video with Hierarchical Variational
  Auto-encoders
  There are many forms of feature information present in video data. Principle
among them are object identity information which is largely static across
multiple video frames, and object pose and style information which continuously
transforms from frame to frame. Most existing models confound these two types
of representation by mapping them to a shared feature space. In this paper we
propose a probabilistic approach for learning separable representations of
object identity and pose information using unsupervised video data. Our
approach leverages a deep generative model with a factored prior distribution
that encodes properties of temporal invariances in the hidden feature set.
Learning is achieved via variational inference. We present results of learning
identity and pose information on a dataset of moving characters as well as a
dataset of rotating 3D objects. Our experimental results demonstrate our
model's success in factoring its representation, and demonstrate that the model
achieves improved performance in transfer learning tasks.
