Hyperparameter Power Impact in Transformer Language Model Training

Lucas Høyberg  Puvis de Chavannes; Mads Guldborg Kjeldgaard Kongsbak; Timmie Rantzau; Leon Derczynski

Hyperparameter Power Impact in Transformer Language Model Training

Lucas Høyberg Puvis de Chavannes, Mads Guldborg Kjeldgaard Kongsbak, Timmie Rantzau, Leon Derczynski

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Original language	English
Title of host publication	Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing
Publisher	Association for Computational Linguistics
Publication date	1 Nov 2021
Publication status	Published - 1 Nov 2021

Access to Document

https://aclanthology.org/2021.sustainlp-1.12.pdfLicence: CC BY

Cite this

@inproceedings{44cde6d2f4054ee896116dc6ad6e124a,

title = "Hyperparameter Power Impact in Transformer Language Model Training",

author = "{Puvis de Chavannes}, {Lucas H{\o}yberg} and Kongsbak, {Mads Guldborg Kjeldgaard} and Timmie Rantzau and Leon Derczynski",

year = "2021",

month = nov,

day = "1",

language = "English",

booktitle = "Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing",

publisher = "Association for Computational Linguistics",

address = "United States",

}

Hyperparameter Power Impact in Transformer Language Model Training. / Puvis de Chavannes, Lucas Høyberg ; Kongsbak, Mads Guldborg Kjeldgaard; Rantzau, Timmie et al.
Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing. Association for Computational Linguistics, 2021.

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

TY - GEN

T1 - Hyperparameter Power Impact in Transformer Language Model Training

AU - Puvis de Chavannes, Lucas Høyberg

AU - Kongsbak, Mads Guldborg Kjeldgaard

AU - Rantzau, Timmie

AU - Derczynski, Leon

PY - 2021/11/1

Y1 - 2021/11/1

M3 - Article in proceedings

BT - Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing

PB - Association for Computational Linguistics

ER -