We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization

Abstract: In the summarization domain, a key requirement for summaries is to be factually consistent with the input document. Previous work has found that natural language inference (NLI) models do not perform competitively when applied to inconsistency detection. In this work, we revisit the use of NLI for inconsistency detection, finding that past work suffered from a mismatch in input granularity between NLI datasets (sentence-level), and inconsistency detection (document level). We provide a highly effective and light-weight method called SummaCConv that enables NLI models to be successfully used for this task by segmenting documents into sentence units and aggregating scores between pairs of sentences. On our newly introduced benchmark called SummaC (Summary Consistency) consisting of six large inconsistency detection datasets, SummaCConv obtains state-of-the-art results with a balanced accuracy of 74.4%, a 5% point improvement compared to prior work. We make the models and datasets available: this https URL
Comments: TACL pre-MIT Press publication version; 11 pages, 2 figures, 5 tables
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2111.09525 [cs.CL]
  (or arXiv:2111.09525v1 [cs.CL] for this version)

Submission history

From: Philippe Laban [view email]
[v1] Thu, 18 Nov 2021 05:02:31 GMT (173kb,D)

Link back to: arXiv, form interface, contact.