Current browse context:
cs.CL
Change to browse by:
References & Citations
Computer Science > Computation and Language
Title: QUACKIE: A NLP Classification Task With Ground Truth Explanations
(Submitted on 24 Dec 2020 (v1), last revised 27 Dec 2020 (this version, v2))
Abstract: NLP Interpretability aims to increase trust in model predictions. This makes evaluating interpretability approaches a pressing issue. There are multiple datasets for evaluating NLP Interpretability, but their dependence on human provided ground truths raises questions about their unbiasedness. In this work, we take a different approach and formulate a specific classification task by diverting question-answering datasets. For this custom classification task, the interpretability ground-truth arises directly from the definition of the classification problem. We use this method to propose a benchmark and lay the groundwork for future research in NLP interpretability by evaluating a wide range of current state of the art methods.
Submission history
From: Yves Rychener [view email][v1] Thu, 24 Dec 2020 10:43:20 GMT (7215kb,D)
[v2] Sun, 27 Dec 2020 18:04:17 GMT (7216kb,D)
Link back to: arXiv, form interface, contact.