Multi-Document Summarization with Centroid-Based Pretraining.

Ratish Surendran Puduppully; Parag Jain; Nancy Chen; Mark Steedman

doi:10.18653/V1/2023.ACL-SHORT.13

Multi-Document Summarization with Centroid-Based Pretraining.

Ratish Surendran Puduppully
, Parag Jain
, Nancy Chen
, Mark Steedman

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Abstract

In Multi-Document Summarization (MDS), the input can be modeled as a set of documents, and the output is its summary. In this paper, we focus on pretraining objectives for MDS. Specifically, we introduce a novel pretraining objective, which involves selecting the ROUGE-based centroid of each document cluster as a proxy for its summary. Our objective thus does not require human written summaries and can be utilized for pretraining on a dataset consisting solely of document sets. Through zero-shot, few-shot, and fully supervised experiments on multiple MDS datasets, we show that our model Centrum is better or comparable to a state-of-the-art model. We make the pretrained and fine-tuned models freely available to the research community

Original language	English
Title of host publication	Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Publisher	Association for Computational Linguistics
Publication date	2023
Pages	128-138
DOIs	https://doi.org/10.18653/V1/2023.ACL-SHORT.13
Publication status	Published - 2023
Externally published	Yes
Event	Annual Meeting of the Association for Computational Linguistics - Toronto, Canada Duration: 9 Jul 2023 → 14 Jul 2023 Conference number: 61 https://aclanthology.org/volumes/2023.acl-short/ https://2023.aclweb.org/

Conference

Conference	Annual Meeting of the Association for Computational Linguistics
Number	61
Country/Territory	Canada
City	Toronto
Period	09/07/2023 → 14/07/2023
Internet address	https://aclanthology.org/volumes/2023.acl-short/ https://2023.aclweb.org/

Keywords

Multi-Document Summarization
Pretraining Objective
ROUGE-based Centroid
Zero-shot Learning
Centrum

Access to Document

10.18653/V1/2023.ACL-SHORT.13

https://dblp.org/rec/conf/acl/PuduppullyJCS23

Cite this

@inproceedings{1394bb0e99174e5bb858c9ae4c98ada5,

title = "Multi-Document Summarization with Centroid-Based Pretraining.",

abstract = "In Multi-Document Summarization (MDS), the input can be modeled as a set of documents, and the output is its summary. In this paper, we focus on pretraining objectives for MDS. Specifically, we introduce a novel pretraining objective, which involves selecting the ROUGE-based centroid of each document cluster as a proxy for its summary. Our objective thus does not require human written summaries and can be utilized for pretraining on a dataset consisting solely of document sets. Through zero-shot, few-shot, and fully supervised experiments on multiple MDS datasets, we show that our model Centrum is better or comparable to a state-of-the-art model. We make the pretrained and fine-tuned models freely available to the research community",

keywords = "Multi-Document Summarization, Pretraining Objective, ROUGE-based Centroid, Zero-shot Learning, Centrum",

author = "Puduppully, \{Ratish Surendran\} and Parag Jain and Nancy Chen and Mark Steedman",

note = "DBLP's bibliographic metadata records provided through http://dblp.org/search/publ/api are distributed under a Creative Commons CC0 1.0 Universal Public Domain Dedication. Although the bibliographic metadata records are provided consistent with CC0 1.0 Dedication, the content described by the metadata records is not. Content may be subject to copyright, rights of privacy, rights of publicity and other restrictions.; Annual Meeting of the Association for Computational Linguistics , ACL ; Conference date: 09-07-2023 Through 14-07-2023",

year = "2023",

doi = "10.18653/V1/2023.ACL-SHORT.13",

language = "English",

pages = "128--138",

booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",

publisher = "Association for Computational Linguistics",

address = "United States",

url = "https://aclanthology.org/volumes/2023.acl-short/, https://2023.aclweb.org/",

}

Puduppully, RS, Jain, P, Chen, N & Steedman, M 2023, Multi-Document Summarization with Centroid-Based Pretraining. in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, pp. 128-138, Annual Meeting of the Association for Computational Linguistics , Toronto, Ontario, Canada, 09/07/2023. https://doi.org/10.18653/V1/2023.ACL-SHORT.13

Multi-Document Summarization with Centroid-Based Pretraining. / Puduppully, Ratish Surendran; Jain, Parag; Chen, Nancy et al.
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, 2023. p. 128-138.

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

TY - GEN

T1 - Multi-Document Summarization with Centroid-Based Pretraining.

AU - Puduppully, Ratish Surendran

AU - Jain, Parag

AU - Chen, Nancy

AU - Steedman, Mark

N1 - Conference code: 61

PY - 2023

Y1 - 2023

N2 - In Multi-Document Summarization (MDS), the input can be modeled as a set of documents, and the output is its summary. In this paper, we focus on pretraining objectives for MDS. Specifically, we introduce a novel pretraining objective, which involves selecting the ROUGE-based centroid of each document cluster as a proxy for its summary. Our objective thus does not require human written summaries and can be utilized for pretraining on a dataset consisting solely of document sets. Through zero-shot, few-shot, and fully supervised experiments on multiple MDS datasets, we show that our model Centrum is better or comparable to a state-of-the-art model. We make the pretrained and fine-tuned models freely available to the research community

AB - In Multi-Document Summarization (MDS), the input can be modeled as a set of documents, and the output is its summary. In this paper, we focus on pretraining objectives for MDS. Specifically, we introduce a novel pretraining objective, which involves selecting the ROUGE-based centroid of each document cluster as a proxy for its summary. Our objective thus does not require human written summaries and can be utilized for pretraining on a dataset consisting solely of document sets. Through zero-shot, few-shot, and fully supervised experiments on multiple MDS datasets, we show that our model Centrum is better or comparable to a state-of-the-art model. We make the pretrained and fine-tuned models freely available to the research community

KW - Multi-Document Summarization

KW - Pretraining Objective

KW - ROUGE-based Centroid

KW - Zero-shot Learning

KW - Centrum

U2 - 10.18653/V1/2023.ACL-SHORT.13

DO - 10.18653/V1/2023.ACL-SHORT.13

M3 - Article in proceedings

SP - 128

EP - 138

BT - Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

PB - Association for Computational Linguistics

T2 - Annual Meeting of the Association for Computational Linguistics

Y2 - 9 July 2023 through 14 July 2023

ER -

Multi-Document Summarization with Centroid-Based Pretraining.

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this