References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Learning to Estimate Shapley Values with Vision Transformers
(Submitted on 10 Jun 2022 (v1), last revised 1 Mar 2023 (this version, v3))
Abstract: Transformers have become a default architecture in computer vision, but understanding what drives their predictions remains a challenging problem. Current explanation approaches rely on attention values or input gradients, but these provide a limited view of a model's dependencies. Shapley values offer a theoretically sound alternative, but their computational cost makes them impractical for large, high-dimensional models. In this work, we aim to make Shapley values practical for vision transformers (ViTs). To do so, we first leverage an attention masking approach to evaluate ViTs with partial information, and we then develop a procedure to generate Shapley value explanations via a separate, learned explainer model. Our experiments compare Shapley values to many baseline methods (e.g., attention rollout, GradCAM, LRP), and we find that our approach provides more accurate explanations than existing methods for ViTs.
Submission history
From: Ian Covert [view email][v1] Fri, 10 Jun 2022 07:09:28 GMT (31586kb,D)
[v2] Fri, 30 Sep 2022 08:49:55 GMT (32547kb,D)
[v3] Wed, 1 Mar 2023 20:24:58 GMT (79801kb,D)
Link back to: arXiv, form interface, contact.