References & Citations
Computer Science > Computation and Language
Title: CO2Sum:Contrastive Learning for Factual-Consistent Abstractive Summarization
(Submitted on 2 Dec 2021 (v1), last revised 9 Jan 2022 (this version, v2))
Abstract: Generating factual-consistent summaries is a challenging task for abstractive summarization. Previous works mainly encode factual information or perform post-correct/rank after decoding. In this paper, we provide a factual-consistent solution from the perspective of contrastive learning, which is a natural extension of previous works. We propose CO2Sum (Contrastive for Consistency), a contrastive learning scheme that can be easily applied on sequence-to-sequence models for factual-consistent abstractive summarization, proving that the model can be fact-aware without modifying the architecture. CO2Sum applies contrastive learning on the encoder, which can help the model be aware of the factual information contained in the input article, or performs contrastive learning on the decoder, which makes the model to generate factual-correct output summary. What's more, these two schemes are orthogonal and can be combined to further improve faithfulness. Comprehensive experiments on public benchmarks demonstrate that CO2Sum improves the faithfulness on large pre-trained language models and reaches competitive results compared to other strong factual-consistent summarization baselines.
Submission history
From: Wei Liu [view email][v1] Thu, 2 Dec 2021 11:52:01 GMT (413kb,D)
[v2] Sun, 9 Jan 2022 07:46:10 GMT (530kb,D)
Link back to: arXiv, form interface, contact.