We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: What GPT Knows About Who is Who

Abstract: Coreference resolution -- which is a crucial task for understanding discourse and language at large -- has yet to witness widespread benefits from large language models (LLMs). Moreover, coreference resolution systems largely rely on supervised labels, which are highly expensive and difficult to annotate, thus making it ripe for prompt engineering. In this paper, we introduce a QA-based prompt-engineering method and discern \textit{generative}, pre-trained LLMs' abilities and limitations toward the task of coreference resolution. Our experiments show that GPT-2 and GPT-Neo can return valid answers, but that their capabilities to identify coreferent mentions are limited and prompt-sensitive, leading to inconsistent results.
Comments: Accepted by ACL 2022 Workshop on Insights from Negative Results in NLP
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:2205.07407 [cs.CL]
  (or arXiv:2205.07407v1 [cs.CL] for this version)

Submission history

From: Xiaohan Yang [view email]
[v1] Mon, 16 May 2022 00:59:37 GMT (952kb,D)

Link back to: arXiv, form interface, contact.