References & Citations
Computer Science > Computation and Language
Title: Interpreting the Robustness of Neural NLP Models to Textual Perturbations
(Submitted on 14 Oct 2021 (v1), last revised 18 Mar 2022 (this version, v2))
Abstract: Modern Natural Language Processing (NLP) models are known to be sensitive to input perturbations and their performance can decrease when applied to real-world, noisy data. However, it is still unclear why models are less robust to some perturbations than others. In this work, we test the hypothesis that the extent to which a model is affected by an unseen textual perturbation (robustness) can be explained by the learnability of the perturbation (defined as how well the model learns to identify the perturbation with a small amount of evidence). We further give a causal justification for the learnability metric. We conduct extensive experiments with four prominent NLP models -- TextRNN, BERT, RoBERTa and XLNet -- over eight types of textual perturbations on three datasets. We show that a model which is better at identifying a perturbation (higher learnability) becomes worse at ignoring such a perturbation at test time (lower robustness), providing empirical support for our hypothesis.
Submission history
From: Yunxiang Zhang [view email][v1] Thu, 14 Oct 2021 05:26:08 GMT (370kb,D)
[v2] Fri, 18 Mar 2022 07:56:11 GMT (1237kb,D)
Link back to: arXiv, form interface, contact.