We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Understanding Model Robustness to User-generated Noisy Texts

Abstract: Sensitivity of deep-neural models to input noise is known to be a challenging problem. In NLP, model performance often deteriorates with naturally occurring noise, such as spelling errors. To mitigate this issue, models may leverage artificially noised data. However, the amount and type of generated noise has so far been determined arbitrarily. We therefore propose to model the errors statistically from grammatical-error-correction corpora. We present a thorough evaluation of several state-of-the-art NLP systems' robustness in multiple languages, with tasks including morpho-syntactic analysis, named entity recognition, neural machine translation, a subset of the GLUE benchmark and reading comprehension. We also compare two approaches to address the performance drop: a) training the NLP models with noised data generated by our framework; and b) reducing the input noise with external system for natural language correction. The code is released at this https URL
Comments: Accepted to W-NUT 2021
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2110.07428 [cs.CL]
  (or arXiv:2110.07428v2 [cs.CL] for this version)

Submission history

From: Milan Straka [view email]
[v1] Thu, 14 Oct 2021 14:54:52 GMT (198kb,D)
[v2] Wed, 17 Nov 2021 18:13:58 GMT (183kb,D)

Link back to: arXiv, form interface, contact.