Current browse context:
cs.CV
Change to browse by:
References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networks
(Submitted on 5 May 2021 (v1), last revised 23 Jun 2021 (this version, v4))
Abstract: Deep neural networks that yield human interpretable decisions by architectural design have lately become an increasingly popular alternative to post hoc interpretation of traditional black-box models. Among these networks, the arguably most widespread approach is so-called prototype learning, where similarities to learned latent prototypes serve as the basis of classifying an unseen data point. In this work, we point to an important shortcoming of such approaches. Namely, there is a semantic gap between similarity in latent space and similarity in input space, which can corrupt interpretability. We design two experiments that exemplify this issue on the so-called ProtoPNet. Specifically, we find that this network's interpretability mechanism can be led astray by intentionally crafted or even JPEG compression artefacts, which can produce incomprehensible decisions. We argue that practitioners ought to have this shortcoming in mind when deploying prototype-based models in practice.
Submission history
From: Jonas Kohler [view email][v1] Wed, 5 May 2021 12:28:34 GMT (14414kb,D)
[v2] Mon, 10 May 2021 08:48:08 GMT (14415kb,D)
[v3] Mon, 21 Jun 2021 10:16:34 GMT (14415kb,D)
[v4] Wed, 23 Jun 2021 12:28:10 GMT (14416kb,D)
Link back to: arXiv, form interface, contact.