We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning

Abstract: To quantitatively and intuitively explore the generalization ability of pre-trained language models (PLMs), we have designed several tasks of arithmetic and logical reasoning. We both analyse how well PLMs generalize when the test data is in the same distribution as the train data and when it is different, for the latter analysis, we have also designed a cross-distribution test set other than the in-distribution test set. We conduct experiments on one of the most advanced and publicly released generative PLM - BART. Our research finds that the PLMs can easily generalize when the distribution is the same, however, it is still difficult for them to generalize out of the distribution.
Comments: Accepted by NLPCC2021
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2108.06743 [cs.CL]
  (or arXiv:2108.06743v2 [cs.CL] for this version)

Submission history

From: Cunxiang Wang [view email]
[v1] Sun, 15 Aug 2021 13:42:10 GMT (11301kb,D)
[v2] Tue, 19 Oct 2021 02:53:26 GMT (11301kb,D)

Link back to: arXiv, form interface, contact.