Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP

Rob van der Goot; Ahmet Üstün; Alan Ramponi; Ibrahim Sharaf; Barbara Plank

doi:10.18653/v1/2021.eacl-demos.22

Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP

Rob van der Goot
, Ahmet Üstün
, Alan Ramponi
, Ibrahim Sharaf
, Barbara Plank

University of Trento
Factmata
University of Groningen

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Abstract

Transfer learning, particularly approaches that combine multi-task learning with pretrained contextualized embeddings and fine-tuning, have advanced the field of Natural Language Processing tremendously in recent years. In this paper we present MaChAmp, a toolkit for easy fine-tuning of contextualized embeddings in multi-task settings. The benefits of MaChAmp are its flexible configuration options, and the support of a variety of natural language processing tasks in a uniform toolkit, from text classification and sequence labeling to dependency parsing, masked language modeling, and text generation.

Original language	English
Title of host publication	Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations
Publisher	Association for Computational Linguistics
Publication date	2021
Pages	176-197
DOIs	https://doi.org/10.18653/v1/2021.eacl-demos.22
Publication status	Published - 2021
Event	Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations - VIRTUAL Duration: 19 Apr 2021 → 23 Apr 2021 Conference number: 16

Conference

Conference	Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations
Number	16
City	VIRTUAL
Period	19/04/2021 → 23/04/2021

Keywords

Transfer Learning
Multi-Task Learning
Pretrained Contextualized Embeddings
Fine-Tuning
Natural Language Processing Toolkit

Access to Document

10.18653/v1/2021.eacl-demos.22

2021.eacl-demos.22Accepted author manuscript, 559 KB

https://www.aclweb.org/anthology/2021.eacl-demos.22/

Outstanding paper award, EACL 2021 demo track
van der Goot, R. (Recipient), Üstün, A. (Recipient), Ramponi, A. (Recipient), Sharaf, I. (Recipient) & Plank, B. (Recipient), 2021
Prize: Prizes, scholarships, distinctions

Multi-Task Sequence Labeling Under Adverse Conditions
Plank, B. (PI) & van der Goot, R. (CoI)
Amazon.com, Inc.
01/04/2019 → 31/08/2020
Project: Other

Cite this

van der Goot, R., Üstün, A., Ramponi, A., Sharaf, I., & Plank, B. (2021). Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations (pp. 176-197). Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.eacl-demos.22

@inproceedings{2ceeedec004545a9921deea5fe4336c1,

title = "Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP",

abstract = "Transfer learning, particularly approaches that combine multi-task learning with pretrained contextualized embeddings and fine-tuning, have advanced the field of Natural Language Processing tremendously in recent years. In this paper we present MaChAmp, a toolkit for easy fine-tuning of contextualized embeddings in multi-task settings. The benefits of MaChAmp are its flexible configuration options, and the support of a variety of natural language processing tasks in a uniform toolkit, from text classification and sequence labeling to dependency parsing, masked language modeling, and text generation.",

keywords = "Transfer Learning, Multi-Task Learning, Pretrained Contextualized Embeddings, Fine-Tuning, Natural Language Processing Toolkit, Transfer Learning, Multi-Task Learning, Pretrained Contextualized Embeddings, Fine-Tuning, Natural Language Processing Toolkit",

author = "\{van der Goot\}, Rob and Ahmet {\"U}st{\"u}n and Alan Ramponi and Ibrahim Sharaf and Barbara Plank",

year = "2021",

doi = "10.18653/v1/2021.eacl-demos.22",

language = "English",

pages = "176--197",

booktitle = "Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations",

publisher = "Association for Computational Linguistics",

address = "United States",

note = "Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations ; Conference date: 19-04-2021 Through 23-04-2021",

}

van der Goot, R, Üstün, A, Ramponi, A, Sharaf, I & Plank, B 2021, Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP. in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics, pp. 176-197, Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, VIRTUAL, 19/04/2021. https://doi.org/10.18653/v1/2021.eacl-demos.22

Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP. / van der Goot, Rob; Üstün, Ahmet; Ramponi, Alan et al.
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics, 2021. p. 176-197.

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

TY - GEN

T1 - Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP

AU - van der Goot, Rob

AU - Üstün, Ahmet

AU - Ramponi, Alan

AU - Sharaf, Ibrahim

AU - Plank, Barbara

N1 - Conference code: 16

PY - 2021

Y1 - 2021

N2 - Transfer learning, particularly approaches that combine multi-task learning with pretrained contextualized embeddings and fine-tuning, have advanced the field of Natural Language Processing tremendously in recent years. In this paper we present MaChAmp, a toolkit for easy fine-tuning of contextualized embeddings in multi-task settings. The benefits of MaChAmp are its flexible configuration options, and the support of a variety of natural language processing tasks in a uniform toolkit, from text classification and sequence labeling to dependency parsing, masked language modeling, and text generation.

AB - Transfer learning, particularly approaches that combine multi-task learning with pretrained contextualized embeddings and fine-tuning, have advanced the field of Natural Language Processing tremendously in recent years. In this paper we present MaChAmp, a toolkit for easy fine-tuning of contextualized embeddings in multi-task settings. The benefits of MaChAmp are its flexible configuration options, and the support of a variety of natural language processing tasks in a uniform toolkit, from text classification and sequence labeling to dependency parsing, masked language modeling, and text generation.

KW - Transfer Learning

KW - Multi-Task Learning

KW - Pretrained Contextualized Embeddings

KW - Fine-Tuning

KW - Natural Language Processing Toolkit

KW - Transfer Learning

KW - Multi-Task Learning

KW - Pretrained Contextualized Embeddings

KW - Fine-Tuning

KW - Natural Language Processing Toolkit

UR - https://machamp-nlp.github.io/

U2 - 10.18653/v1/2021.eacl-demos.22

DO - 10.18653/v1/2021.eacl-demos.22

M3 - Article in proceedings

SP - 176

EP - 197

BT - Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations

PB - Association for Computational Linguistics

T2 - Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations

Y2 - 19 April 2021 through 23 April 2021

ER -

Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP

Abstract

Conference

Keywords

Access to Document

Fingerprint

Prizes

Outstanding paper award, EACL 2021 demo track

Projects

Multi-Task Sequence Labeling Under Adverse Conditions

Cite this