We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures

Abstract: While mainstream machine learning methods are known to have limited ability to compositionally generalize, new architectures and techniques continue to be proposed to address this limitation. We investigate state-of-the-art techniques and architectures in order to assess their effectiveness in improving compositional generalization in semantic parsing tasks based on the SCAN and CFQ datasets. We show that masked language model (MLM) pre-training rivals SCAN-inspired architectures on primitive holdout splits. On a more complex compositional task, we show that pre-training leads to significant improvements in performance vs. comparable non-pre-trained models, whereas architectures proposed to encourage compositional generalization on SCAN or in the area of algorithm learning fail to lead to significant improvements. We establish a new state of the art on the CFQ compositional generalization benchmark using MLM pre-training together with an intermediate representation.
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:2007.08970 [cs.CL]
  (or arXiv:2007.08970v3 [cs.CL] for this version)

Submission history

From: Marc van Zee [view email]
[v1] Fri, 17 Jul 2020 13:34:49 GMT (981kb,D)
[v2] Tue, 21 Jul 2020 15:31:05 GMT (981kb,D)
[v3] Wed, 22 Sep 2021 09:06:43 GMT (461kb,D)

Link back to: arXiv, form interface, contact.