Noise-Aware Video Saliency Prediction
  We tackle the problem of predicting saliency maps for videos of dynamic
scenes. We note that the accuracy of the maps reconstructed from the gaze data
of a fixed number of observers varies with the frame, as it depends on the
content of the scene. This issue is particularly pressing when a limited number
of observers are available. In such cases, directly minimizing the discrepancy
between the predicted and measured saliency maps, as traditional deep-learning
methods do, results in overfitting to the noisy data. We propose a noise-aware
training (NAT) paradigm that quantifies and accounts for the uncertainty
arising from frame-specific gaze data inaccuracy. We show that NAT is
especially advantageous when limited training data is available, with
experiments across different models, loss functions, and datasets. We also
introduce a video game-based saliency dataset, with rich temporal semantics,
and multiple gaze attractors per frame. The dataset and source code are
available at https://github.com/NVlabs/NAT-saliency.
