We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Computer Science > Computation and Language

Title: SWING: Balancing Coverage and Faithfulness for Dialogue Summarization

Abstract: Missing information is a common issue of dialogue summarization where some information in the reference summaries is not covered in the generated summaries. To address this issue, we propose to utilize natural language inference (NLI) models to improve coverage while avoiding introducing factual inconsistencies. Specifically, we use NLI to compute fine-grained training signals to encourage the model to generate content in the reference summaries that have not been covered, as well as to distinguish between factually consistent and inconsistent generated sentences. Experiments on the DialogSum and SAMSum datasets confirm the effectiveness of the proposed approach in balancing coverage and faithfulness, validated with automatic metrics and human evaluations. Additionally, we compute the correlation between commonly used automatic metrics with human judgments in terms of three different dimensions regarding coverage and factual consistency to provide insight into the most suitable metric for evaluating dialogue summaries.
Comments: Accepted by Findings of EACL 2023
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2301.10483 [cs.CL]
  (or arXiv:2301.10483v1 [cs.CL] for this version)

Submission history

From: Kung-Hsiang Huang [view email]
[v1] Wed, 25 Jan 2023 09:33:11 GMT (248kb,D)

Link back to: arXiv, form interface, contact.