References & Citations
Computer Science > Computation and Language
Title: COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization
(Submitted on 29 Sep 2022 (v1), last revised 19 Apr 2023 (this version, v2))
Abstract: Traditional training paradigms for extractive and abstractive summarization systems always only use token-level or sentence-level training objectives. However, the output summary is always evaluated from summary-level which leads to the inconsistency in training and evaluation. In this paper, we propose a Contrastive Learning based re-ranking framework for one-stage summarization called COLO. By modeling a contrastive objective, we show that the summarization model is able to directly generate summaries according to the summary-level score without additional modules and parameters. Extensive experiments demonstrate that COLO boosts the extractive and abstractive results of one-stage systems on CNN/DailyMail benchmark to 44.58 and 46.33 ROUGE-1 score while preserving the parameter efficiency and inference efficiency. Compared with state-of-the-art multi-stage systems, we save more than 100 GPU training hours and obtaining 3~8 speed-up ratio during inference while maintaining comparable results.
Submission history
From: Chenxin An [view email][v1] Thu, 29 Sep 2022 06:11:21 GMT (1285kb,D)
[v2] Wed, 19 Apr 2023 07:01:18 GMT (1285kb,D)
Link back to: arXiv, form interface, contact.