References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Noise-Aware Video Saliency Prediction
(Submitted on 16 Apr 2021 (v1), last revised 22 Nov 2021 (this version, v2))
Abstract: We tackle the problem of predicting saliency maps for videos of dynamic scenes. We note that the accuracy of the maps reconstructed from the gaze data of a fixed number of observers varies with the frame, as it depends on the content of the scene. This issue is particularly pressing when a limited number of observers are available. In such cases, directly minimizing the discrepancy between the predicted and measured saliency maps, as traditional deep-learning methods do, results in overfitting to the noisy data. We propose a noise-aware training (NAT) paradigm that quantifies and accounts for the uncertainty arising from frame-specific gaze data inaccuracy. We show that NAT is especially advantageous when limited training data is available, with experiments across different models, loss functions, and datasets. We also introduce a video game-based saliency dataset, with rich temporal semantics, and multiple gaze attractors per frame. The dataset and source code are available at this https URL
Submission history
From: Ekta Prashnani [view email][v1] Fri, 16 Apr 2021 11:32:46 GMT (25457kb,D)
[v2] Mon, 22 Nov 2021 05:38:36 GMT (27591kb,D)
Link back to: arXiv, form interface, contact.