References & Citations
Computer Science > Computation and Language
Title: Transforming Wikipedia into Augmented Data for Query-Focused Summarization
(Submitted on 8 Nov 2019 (v1), last revised 22 Jul 2022 (this version, v2))
Abstract: The limited size of existing query-focused summarization datasets renders training data-driven summarization models challenging. Meanwhile, the manual construction of a query-focused summarization corpus is costly and time-consuming. In this paper, we use Wikipedia to automatically collect a large query-focused summarization dataset (named WIKIREF) of more than 280, 000 examples, which can serve as a means of data augmentation. We also develop a BERT-based query-focused summarization model (Q-BERT) to extract sentences from the documents as summaries. To better adapt a huge model containing millions of parameters to tiny benchmarks, we identify and fine-tune only a sparse subnetwork, which corresponds to a small fraction of the whole model parameters. Experimental results on three DUC benchmarks show that the model pre-trained on WIKIREF has already achieved reasonable performance. After fine-tuning on the specific benchmark datasets, the model with data augmentation outperforms strong comparison systems. Moreover, both our proposed Q-BERT model and subnetwork fine-tuning further improve the model performance. The dataset is publicly available at this https URL
Submission history
From: Haichao Zhu [view email][v1] Fri, 8 Nov 2019 15:28:21 GMT (1054kb,D)
[v2] Fri, 22 Jul 2022 15:47:29 GMT (1862kb,D)
Link back to: arXiv, form interface, contact.