References & Citations
Computer Science > Computation and Language
Title: Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports
(Submitted on 6 Nov 2019 (v1), last revised 28 Apr 2020 (this version, v3))
Abstract: Neural abstractive summarization models are able to generate summaries which have high overlap with human references. However, existing models are not optimized for factual correctness, a critical metric in real-world applications. In this work, we develop a general framework where we evaluate the factual correctness of a generated summary by fact-checking it automatically against its reference using an information extraction module. We further propose a training strategy which optimizes a neural summarization model with a factual correctness reward via reinforcement learning. We apply the proposed method to the summarization of radiology reports, where factual correctness is a key requirement. On two separate datasets collected from hospitals, we show via both automatic and human evaluation that the proposed approach substantially improves the factual correctness and overall quality of outputs over a competitive neural summarization system, producing radiology summaries that approach the quality of human-authored ones.
Submission history
From: Yuhao Zhang [view email][v1] Wed, 6 Nov 2019 18:25:00 GMT (371kb,D)
[v2] Fri, 8 Nov 2019 07:39:03 GMT (369kb,D)
[v3] Tue, 28 Apr 2020 01:33:49 GMT (348kb,D)
Link back to: arXiv, form interface, contact.