Deep multi-scale video prediction beyond mean square error
  Learning to predict future images from a video sequence involves the
construction of an internal representation that models the image evolution
accurately, and therefore, to some degree, its content and dynamics. This is
why pixel-space video prediction may be viewed as a promising avenue for
unsupervised feature learning. In addition, while optical flow has been a very
studied problem in computer vision for a long time, future frame prediction is
rarely approached. Still, many vision applications could benefit from the
knowledge of the next frames of videos, that does not require the complexity of
tracking every pixel trajectories. In this work, we train a convolutional
network to generate future frames given an input sequence. To deal with the
inherently blurry predictions obtained from the standard Mean Squared Error
(MSE) loss function, we propose three different and complementary feature
learning strategies: a multi-scale architecture, an adversarial training
method, and an image gradient difference loss function. We compare our
predictions to different published results based on recurrent neural networks
on the UCF101 dataset
