We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: On the Lack of Robust Interpretability of Neural Text Classifiers

Abstract: With the ever-increasing complexity of neural language models, practitioners have turned to methods for understanding the predictions of these models. One of the most well-adopted approaches for model interpretability is feature-based interpretability, i.e., ranking the features in terms of their impact on model predictions. Several prior studies have focused on assessing the fidelity of feature-based interpretability methods, i.e., measuring the impact of dropping the top-ranked features on the model output. However, relatively little work has been conducted on quantifying the robustness of interpretations. In this work, we assess the robustness of interpretations of neural text classifiers, specifically, those based on pretrained Transformer encoders, using two randomization tests. The first compares the interpretations of two models that are identical except for their initializations. The second measures whether the interpretations differ between a model with trained parameters and a model with random parameters. Both tests show surprising deviations from expected behavior, raising questions about the extent of insights that practitioners may draw from interpretations.
Comments: Appearing at ACL Findings 2021
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:2106.04631 [cs.CL]
  (or arXiv:2106.04631v1 [cs.CL] for this version)

Submission history

From: Muhammad Bilal Zafar [view email]
[v1] Tue, 8 Jun 2021 18:31:02 GMT (52kb,D)

Link back to: arXiv, form interface, contact.