We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Adversarial Training for Machine Reading Comprehension with Virtual Embeddings

Abstract: Adversarial training (AT) as a regularization method has proved its effectiveness on various tasks. Though there are successful applications of AT on some NLP tasks, the distinguishing characteristics of NLP tasks have not been exploited. In this paper, we aim to apply AT on machine reading comprehension (MRC) tasks. Furthermore, we adapt AT for MRC tasks by proposing a novel adversarial training method called PQAT that perturbs the embedding matrix instead of word vectors. To differentiate the roles of passages and questions, PQAT uses additional virtual P/Q-embedding matrices to gather the global perturbations of words from passages and questions separately. We test the method on a wide range of MRC tasks, including span-based extractive RC and multiple-choice RC. The results show that adversarial training is effective universally, and PQAT further improves the performance.
Comments: Accepted to *SEM 2021 workshop at ACL 2021
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2106.04437 [cs.CL]
  (or arXiv:2106.04437v1 [cs.CL] for this version)

Submission history

From: Ziqing Yang [view email]
[v1] Tue, 8 Jun 2021 15:16:34 GMT (183kb,D)

Link back to: arXiv, form interface, contact.