A Hierarchical Variational Neural Uncertainty Model for Stochastic Video
  Prediction
  Predicting the future frames of a video is a challenging task, in part due to
the underlying stochastic real-world phenomena. Prior approaches to solve this
task typically estimate a latent prior characterizing this stochasticity,
however do not account for the predictive uncertainty of the (deep learning)
model. Such approaches often derive the training signal from the mean-squared
error (MSE) between the generated frame and the ground truth, which can lead to
sub-optimal training, especially when the predictive uncertainty is high.
Towards this end, we introduce Neural Uncertainty Quantifier (NUQ) - a
stochastic quantification of the model's predictive uncertainty, and use it to
weigh the MSE loss. We propose a hierarchical, variational framework to derive
NUQ in a principled manner using a deep, Bayesian graphical model. Our
experiments on four benchmark stochastic video prediction datasets show that
our proposed framework trains more effectively compared to the state-of-the-art
models (especially when the training sets are small), while demonstrating
better video generation quality and diversity against several evaluation
metrics.
