We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CV

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computer Vision and Pattern Recognition

Title: Comparing object recognition in humans and deep convolutional neural networks -- An eye tracking study

Abstract: Deep convolutional neural networks (DCNNs) and the ventral visual pathway share vast architectural and functional similarities in visual challenges such as object recognition. Recent insights have demonstrated that both hierarchical cascades can be compared in terms of both exerted behavior and underlying activation. However, these approaches ignore key differences in spatial priorities of information processing. In this proof-of-concept study, we demonstrate a comparison of human observers (N = 45) and three feedforward DCNNs through eye tracking and saliency maps. The results reveal fundamentally different resolutions in both visualization methods that need to be considered for an insightful comparison. Moreover, we provide evidence that a DCNN with biologically plausible receptive field sizes called vNet reveals higher agreement with human viewing behavior as contrasted with a standard ResNet architecture. We find that image-specific factors such as category, animacy, arousal, and valence have a direct link to the agreement of spatial object recognition priorities in humans and DCNNs, while other measures such as difficulty and general image properties do not. With this approach, we try to open up new perspectives at the intersection of biological and computer vision research.
Comments: 25 pages, 10 figures, 1 table
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:2108.00107 [cs.CV]
  (or arXiv:2108.00107v2 [cs.CV] for this version)

Submission history

From: Leonard Elia van Dyck [view email]
[v1] Fri, 30 Jul 2021 23:32:05 GMT (1322kb)
[v2] Tue, 21 Sep 2021 13:49:26 GMT (1777kb)

Link back to: arXiv, form interface, contact.