Current browse context:
cs.CL
Change to browse by:
References & Citations
Computer Science > Computation and Language
Title: Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
(Submitted on 27 Jul 2019 (v1), last revised 8 Apr 2020 (this version, v6))
Abstract: Machine learning algorithms are often vulnerable to adversarial examples that have imperceptible alterations from the original counterparts but can fool the state-of-the-art models. It is helpful to evaluate or even improve the robustness of these models by exposing the maliciously crafted adversarial examples. In this paper, we present TextFooler, a simple but strong baseline to generate natural adversarial text. By applying it to two fundamental natural language tasks, text classification and textual entailment, we successfully attacked three target models, including the powerful pre-trained BERT, and the widely used convolutional and recurrent neural networks. We demonstrate the advantages of this framework in three ways: (1) effective---it outperforms state-of-the-art attacks in terms of success rate and perturbation rate, (2) utility-preserving---it preserves semantic content and grammaticality, and remains correctly classified by humans, and (3) efficient---it generates adversarial text with computational complexity linear to the text length. *The code, pre-trained target models, and test examples are available at this https URL
Submission history
From: Zhijing Jin [view email][v1] Sat, 27 Jul 2019 15:07:04 GMT (170kb)
[v2] Sat, 7 Sep 2019 05:33:35 GMT (174kb)
[v3] Sat, 23 Nov 2019 07:53:08 GMT (280kb)
[v4] Thu, 23 Jan 2020 07:16:25 GMT (34kb)
[v5] Sun, 5 Apr 2020 07:12:08 GMT (35kb)
[v6] Wed, 8 Apr 2020 23:10:10 GMT (969kb,D)
Link back to: arXiv, form interface, contact.