Transformation-Based Models of Video Sequences
  In this work we propose a simple unsupervised approach for next frame
prediction in video. Instead of directly predicting the pixels in a frame given
past frames, we predict the transformations needed for generating the next
frame in a sequence, given the transformations of the past frames. This leads
to sharper results, while using a smaller prediction model.
  In order to enable a fair comparison between different video frame prediction
models, we also propose a new evaluation protocol. We use generated frames as
input to a classifier trained with ground truth sequences. This criterion
guarantees that models scoring high are those producing sequences which
preserve discrim- inative features, as opposed to merely penalizing any
deviation, plausible or not, from the ground truth. Our proposed approach
compares favourably against more sophisticated ones on the UCF-101 data set,
while also being more efficient in terms of the number of parameters and
computational cost.
