We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Entailment Semantics Can Be Extracted from an Ideal Language Model

Abstract: Language models are often trained on text alone, without additional grounding. There is debate as to how much of natural language semantics can be inferred from such a procedure. We prove that entailment judgments between sentences can be extracted from an ideal language model that has perfectly learned its target distribution, assuming the training sentences are generated by Gricean agents, i.e., agents who follow fundamental principles of communication from the linguistic theory of pragmatics. We also show entailment judgments can be decoded from the predictions of a language model trained on such Gricean data. Our results reveal a pathway for understanding the semantic information encoded in unlabeled linguistic data and a potential framework for extracting semantics from language models.
Comments: Accepted at CONLL 2022
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2209.12407 [cs.CL]
  (or arXiv:2209.12407v1 [cs.CL] for this version)

Submission history

From: William Merrill [view email]
[v1] Mon, 26 Sep 2022 04:16:02 GMT (110kb,D)

Link back to: arXiv, form interface, contact.