We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CV

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computer Vision and Pattern Recognition

Title: Understanding Failures of Deep Networks via Robust Feature Extraction

Abstract: Traditional evaluation metrics for learned models that report aggregate scores over a test set are insufficient for surfacing important and informative patterns of failure over features and instances. We introduce and study a method aimed at characterizing and explaining failures by identifying visual attributes whose presence or absence results in poor performance. In distinction to previous work that relies upon crowdsourced labels for visual attributes, we leverage the representation of a separate robust model to extract interpretable features and then harness these features to identify failure modes. We further propose a visualization method aimed at enabling humans to understand the meaning encoded in such features and we test the comprehensibility of the features. An evaluation of the methods on the ImageNet dataset demonstrates that: (i) the proposed workflow is effective for discovering important failure modes, (ii) the visualization techniques help humans to understand the extracted features, and (iii) the extracted insights can assist engineers with error analysis and debugging.
Comments: Accepted at CVPR, 2021
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:2012.01750 [cs.CV]
  (or arXiv:2012.01750v3 [cs.CV] for this version)

Submission history

From: Sahil Singla [view email]
[v1] Thu, 3 Dec 2020 08:33:29 GMT (54212kb,D)
[v2] Thu, 13 May 2021 12:44:02 GMT (18803kb,D)
[v3] Sat, 12 Jun 2021 07:15:51 GMT (18803kb,D)

Link back to: arXiv, form interface, contact.