Current browse context:
cs.CV
Change to browse by:
References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: The effectiveness of feature attribution methods and its correlation with automatic evaluation scores
(Submitted on 31 May 2021 (v1), last revised 22 Jul 2022 (this version, v5))
Abstract: Explaining the decisions of an Artificial Intelligence (AI) model is increasingly critical in many real-world, high-stake applications. Hundreds of papers have either proposed new feature attribution methods, discussed or harnessed these tools in their work. However, despite humans being the target end-users, most attribution methods were only evaluated on proxy automatic-evaluation metrics (Zhang et al. 2018; Zhou et al. 2016; Petsiuk et al. 2018). In this paper, we conduct the first user study to measure attribution map effectiveness in assisting humans in ImageNet classification and Stanford Dogs fine-grained classification, and when an image is natural or adversarial (i.e., contains adversarial perturbations). Overall, feature attribution is surprisingly not more effective than showing humans nearest training-set examples. On a harder task of fine-grained dog categorization, presenting attribution maps to humans does not help, but instead hurts the performance of human-AI teams compared to AI alone. Importantly, we found automatic attribution-map evaluation measures to correlate poorly with the actual human-AI team performance. Our findings encourage the community to rigorously test their methods on the downstream human-in-the-loop applications and to rethink the existing evaluation metrics.
Submission history
From: Anh Nguyen [view email][v1] Mon, 31 May 2021 13:23:50 GMT (21732kb,D)
[v2] Wed, 23 Jun 2021 22:00:51 GMT (21733kb,D)
[v3] Thu, 28 Oct 2021 11:19:44 GMT (21737kb,D)
[v4] Thu, 27 Jan 2022 15:55:55 GMT (21737kb,D)
[v5] Fri, 22 Jul 2022 05:44:50 GMT (21737kb,D)
Link back to: arXiv, form interface, contact.