Multi-Document Summarization with Centroid-Based Pretraining.

Ratish Surendran Puduppully; Parag Jain; Nancy Chen; Mark Steedman

doi:10.18653/V1/2023.ACL-SHORT.13

Multi-Document Summarization with Centroid-Based Pretraining.

Ratish Surendran Puduppully
, Parag Jain
, Nancy Chen
, Mark Steedman

Agency for Science, Technology and Research (A*Star)
University of Edinburgh

Publikation: Konference artikel i Proceeding eller bog/rapport kapitel › Konferencebidrag i proceedings › Forskning › peer review

Abstract

In Multi-Document Summarization (MDS), the input can be modeled as a set of documents, and the output is its summary. In this paper, we focus on pretraining objectives for MDS. Specifically, we introduce a novel pretraining objective, which involves selecting the ROUGE-based centroid of each document cluster as a proxy for its summary. Our objective thus does not require human written summaries and can be utilized for pretraining on a dataset consisting solely of document sets. Through zero-shot, few-shot, and fully supervised experiments on multiple MDS datasets, we show that our model Centrum is better or comparable to a state-of-the-art model. We make the pretrained and fine-tuned models freely available to the research community

Originalsprog	Engelsk
Titel	Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Forlag	Association for Computational Linguistics
Publikationsdato	2023
Sider	128-138
DOI	https://doi.org/10.18653/V1/2023.ACL-SHORT.13
Status	Udgivet - 2023
Udgivet eksternt	Ja
Begivenhed	Meeting of the Association for Computational Linguistics - Toronto, Canada Varighed: 9 jul. 2023 → 14 jul. 2023 Konferencens nummer: 61 https://aclanthology.org/volumes/2023.acl-short/ https://2023.aclweb.org/

Konference

Konference	Meeting of the Association for Computational Linguistics
Nummer	61
Land/Område	Canada
By	Toronto
Periode	09/07/2023 → 14/07/2023
Internetadresse	https://aclanthology.org/volumes/2023.acl-short/ https://2023.aclweb.org/

Adgang til dokumentet

10.18653/V1/2023.ACL-SHORT.13

https://dblp.org/rec/conf/acl/PuduppullyJCS23

Citationsformater

@inproceedings{1394bb0e99174e5bb858c9ae4c98ada5,

title = "Multi-Document Summarization with Centroid-Based Pretraining.",

abstract = "In Multi-Document Summarization (MDS), the input can be modeled as a set of documents, and the output is its summary. In this paper, we focus on pretraining objectives for MDS. Specifically, we introduce a novel pretraining objective, which involves selecting the ROUGE-based centroid of each document cluster as a proxy for its summary. Our objective thus does not require human written summaries and can be utilized for pretraining on a dataset consisting solely of document sets. Through zero-shot, few-shot, and fully supervised experiments on multiple MDS datasets, we show that our model Centrum is better or comparable to a state-of-the-art model. We make the pretrained and fine-tuned models freely available to the research community",

keywords = "Multi-Document Summarization, Pretraining Objective, ROUGE-based Centroid, Zero-shot Learning, Centrum",

author = "Puduppully, \{Ratish Surendran\} and Parag Jain and Nancy Chen and Mark Steedman",

note = "DBLP's bibliographic metadata records provided through http://dblp.org/search/publ/api are distributed under a Creative Commons CC0 1.0 Universal Public Domain Dedication. Although the bibliographic metadata records are provided consistent with CC0 1.0 Dedication, the content described by the metadata records is not. Content may be subject to copyright, rights of privacy, rights of publicity and other restrictions.; Annual Meeting of the Association for Computational Linguistics , ACL ; Conference date: 09-07-2023 Through 14-07-2023",

year = "2023",

doi = "10.18653/V1/2023.ACL-SHORT.13",

language = "English",

pages = "128--138",

booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",

publisher = "Association for Computational Linguistics",

address = "United States",

url = "https://aclanthology.org/volumes/2023.acl-short/, https://2023.aclweb.org/",

}

Puduppully, RS, Jain, P, Chen, N & Steedman, M 2023, Multi-Document Summarization with Centroid-Based Pretraining. i Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, s. 128-138, Meeting of the Association for Computational Linguistics, Toronto, Ontario, Canada, 09/07/2023. https://doi.org/10.18653/V1/2023.ACL-SHORT.13

Multi-Document Summarization with Centroid-Based Pretraining. / Puduppully, Ratish Surendran; Jain, Parag; Chen, Nancy et al.
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, 2023. s. 128-138.

Publikation: Konference artikel i Proceeding eller bog/rapport kapitel › Konferencebidrag i proceedings › Forskning › peer review

TY - GEN

T1 - Multi-Document Summarization with Centroid-Based Pretraining.

AU - Puduppully, Ratish Surendran

AU - Jain, Parag

AU - Chen, Nancy

AU - Steedman, Mark

N1 - Conference code: 61

PY - 2023

Y1 - 2023

N2 - In Multi-Document Summarization (MDS), the input can be modeled as a set of documents, and the output is its summary. In this paper, we focus on pretraining objectives for MDS. Specifically, we introduce a novel pretraining objective, which involves selecting the ROUGE-based centroid of each document cluster as a proxy for its summary. Our objective thus does not require human written summaries and can be utilized for pretraining on a dataset consisting solely of document sets. Through zero-shot, few-shot, and fully supervised experiments on multiple MDS datasets, we show that our model Centrum is better or comparable to a state-of-the-art model. We make the pretrained and fine-tuned models freely available to the research community

AB - In Multi-Document Summarization (MDS), the input can be modeled as a set of documents, and the output is its summary. In this paper, we focus on pretraining objectives for MDS. Specifically, we introduce a novel pretraining objective, which involves selecting the ROUGE-based centroid of each document cluster as a proxy for its summary. Our objective thus does not require human written summaries and can be utilized for pretraining on a dataset consisting solely of document sets. Through zero-shot, few-shot, and fully supervised experiments on multiple MDS datasets, we show that our model Centrum is better or comparable to a state-of-the-art model. We make the pretrained and fine-tuned models freely available to the research community

KW - Multi-Document Summarization

KW - Pretraining Objective

KW - ROUGE-based Centroid

KW - Zero-shot Learning

KW - Centrum

U2 - 10.18653/V1/2023.ACL-SHORT.13

DO - 10.18653/V1/2023.ACL-SHORT.13

M3 - Article in proceedings

SP - 128

EP - 138

BT - Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

PB - Association for Computational Linguistics

T2 - Annual Meeting of the Association for Computational Linguistics

Y2 - 9 July 2023 through 14 July 2023

ER -

Multi-Document Summarization with Centroid-Based Pretraining.

Abstract

Konference

Adgang til dokumentet

Fingeraftryk

Citationsformater