We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Evaluating Explanations: How much do explanations from the teacher aid students?

Abstract: While many methods purport to explain predictions by highlighting salient features, what aims these explanations serve and how they ought to be evaluated often go unstated. In this work, we introduce a framework to quantify the value of explanations via the accuracy gains that they confer on a student model trained to simulate a teacher model. Crucially, the explanations are available to the student during training, but are not available at test time. Compared to prior proposals, our approach is less easily gamed, enabling principled, automatic, model-agnostic evaluation of attributions. Using our framework, we compare numerous attribution methods for text classification and question answering, and observe quantitative differences that are consistent (to a moderate to high degree) across different student model architectures and learning strategies.
Comments: TACL 2021 (pre-MIT Press publication version)
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:2012.00893 [cs.CL]
  (or arXiv:2012.00893v2 [cs.CL] for this version)

Submission history

From: Danish Pruthi [view email]
[v1] Tue, 1 Dec 2020 23:40:21 GMT (633kb,D)
[v2] Fri, 17 Dec 2021 04:50:55 GMT (494kb,D)

Link back to: arXiv, form interface, contact.