Program Chairs' Report on Peer Review at ACL 2023

Anna Rogers; Marzena Karpinska; Jordan Boyd-Graber; Naoaki Okazaki

Program Chairs' Report on Peer Review at ACL 2023

Anna Rogers
, Marzena Karpinska
, Jordan Boyd-Graber
, Naoaki Okazaki

NLPnorth

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research

Abstract

We present a summary of the efforts to improve conference peer review that were implemented at ACL'23. This includes work with the goal of improving review quality, clearer workflow and decision support for the area chairs, as well as our efforts to improve paper-reviewer matching for various kinds of non- mainstream NLP work, and improve the overall incentives for all participants of the peer review process. We present analysis of the factors affecting peer review, identify the most problematic issues that the authors complained about, and provide suggestions for the future chairs. We hope that publishing such reports would (a) improve transparency in decision-making, (b) help the people new to the field to understand how the *ACL conferences work, (c) provide useful data for the future chairs and workshop organizers, and also academic work on peer review, and (d) provide useful context for the final program, as a source of information for meta-research on the structure and trajectory of the field of NLP.

Original language	English
Title of host publication	Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Place of Publication	Toronto, Canada
Publisher	Association for Computational Linguistics
Publication date	1 Jul 2023
Pages	xl-lxxv
Publication status	Published - 1 Jul 2023
Event	Annual Meeting of the Association for Computational Linguistics - Toronto, Canada Duration: 9 Jul 2023 → 14 Jul 2023 Conference number: 61 https://aclanthology.org/volumes/2023.acl-short/ https://2023.aclweb.org/

Conference

Conference	Annual Meeting of the Association for Computational Linguistics
Number	61
Country/Territory	Canada
City	Toronto
Period	09/07/2023 → 14/07/2023
Internet address	https://aclanthology.org/volumes/2023.acl-short/ https://2023.aclweb.org/

Keywords

Peer Review
Conference Management
Review Quality
NLP
Peer Review Matching
Transparency
Decision Support
Incentives
Meta-Research
ACL'23

Access to Document

Cite this

@inproceedings{7407b13717704be4bb040878dfd6fba0,

title = "Program Chairs' Report on Peer Review at ACL 2023",

abstract = "We present a summary of the efforts to improve conference peer review that were implemented at ACL'23. This includes work with the goal of improving review quality, clearer workflow and decision support for the area chairs, as well as our efforts to improve paper-reviewer matching for various kinds of non- mainstream NLP work, and improve the overall incentives for all participants of the peer review process. We present analysis of the factors affecting peer review, identify the most problematic issues that the authors complained about, and provide suggestions for the future chairs. We hope that publishing such reports would (a) improve transparency in decision-making, (b) help the people new to the field to understand how the *ACL conferences work, (c) provide useful data for the future chairs and workshop organizers, and also academic work on peer review, and (d) provide useful context for the final program, as a source of information for meta-research on the structure and trajectory of the field of NLP.",

keywords = "Peer Review, Conference Management, Review Quality, NLP, Peer Review Matching, Transparency, Decision Support, Incentives, Meta-Research, ACL'23, Peer Review, Conference Management, Review Quality, NLP, Peer Review Matching, Transparency, Decision Support, Incentives, Meta-Research, ACL'23",

author = "Anna Rogers and Marzena Karpinska and Jordan Boyd-Graber and Naoaki Okazaki",

year = "2023",

month = jul,

day = "1",

language = "English",

pages = "xl--lxxv",

booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",

publisher = "Association for Computational Linguistics",

address = "United States",

note = "Annual Meeting of the Association for Computational Linguistics , ACL ; Conference date: 09-07-2023 Through 14-07-2023",

url = "https://aclanthology.org/volumes/2023.acl-short/, https://2023.aclweb.org/",

}

Rogers, A, Karpinska, M, Boyd-Graber, J & Okazaki, N 2023, Program Chairs' Report on Peer Review at ACL 2023. in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Toronto, Canada, pp. xl-lxxv, Annual Meeting of the Association for Computational Linguistics , Toronto, Ontario, Canada, 09/07/2023. <https://aclanthology.org/2023.acl-long.911/>

Program Chairs' Report on Peer Review at ACL 2023. / Rogers, Anna; Karpinska, Marzena; Boyd-Graber, Jordan et al.
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: Association for Computational Linguistics, 2023. p. xl-lxxv.

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research

TY - GEN

T1 - Program Chairs' Report on Peer Review at ACL 2023

AU - Rogers, Anna

AU - Karpinska, Marzena

AU - Boyd-Graber, Jordan

AU - Okazaki, Naoaki

N1 - Conference code: 61

PY - 2023/7/1

Y1 - 2023/7/1

N2 - We present a summary of the efforts to improve conference peer review that were implemented at ACL'23. This includes work with the goal of improving review quality, clearer workflow and decision support for the area chairs, as well as our efforts to improve paper-reviewer matching for various kinds of non- mainstream NLP work, and improve the overall incentives for all participants of the peer review process. We present analysis of the factors affecting peer review, identify the most problematic issues that the authors complained about, and provide suggestions for the future chairs. We hope that publishing such reports would (a) improve transparency in decision-making, (b) help the people new to the field to understand how the *ACL conferences work, (c) provide useful data for the future chairs and workshop organizers, and also academic work on peer review, and (d) provide useful context for the final program, as a source of information for meta-research on the structure and trajectory of the field of NLP.

AB - We present a summary of the efforts to improve conference peer review that were implemented at ACL'23. This includes work with the goal of improving review quality, clearer workflow and decision support for the area chairs, as well as our efforts to improve paper-reviewer matching for various kinds of non- mainstream NLP work, and improve the overall incentives for all participants of the peer review process. We present analysis of the factors affecting peer review, identify the most problematic issues that the authors complained about, and provide suggestions for the future chairs. We hope that publishing such reports would (a) improve transparency in decision-making, (b) help the people new to the field to understand how the *ACL conferences work, (c) provide useful data for the future chairs and workshop organizers, and also academic work on peer review, and (d) provide useful context for the final program, as a source of information for meta-research on the structure and trajectory of the field of NLP.

KW - Peer Review

KW - Conference Management

KW - Review Quality

KW - NLP

KW - Peer Review Matching

KW - Transparency

KW - Decision Support

KW - Incentives

KW - Meta-Research

KW - ACL'23

KW - Peer Review

KW - Conference Management

KW - Review Quality

KW - NLP

KW - Peer Review Matching

KW - Transparency

KW - Decision Support

KW - Incentives

KW - Meta-Research

KW - ACL'23

M3 - Article in proceedings

SP - xl-lxxv

BT - Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

PB - Association for Computational Linguistics

CY - Toronto, Canada

T2 - Annual Meeting of the Association for Computational Linguistics

Y2 - 9 July 2023 through 14 July 2023

ER -

Program Chairs' Report on Peer Review at ACL 2023

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this