Hyperparameter Power Impact in Transformer Language Model Training

Lucas Høyberg  Puvis de Chavannes; Mads Guldborg Kjeldgaard Kongsbak; Timmie Rantzau; Leon Derczynski

Hyperparameter Power Impact in Transformer Language Model Training

Lucas Høyberg Puvis de Chavannes
, Mads Guldborg Kjeldgaard Kongsbak
, Timmie Rantzau
, Leon Derczynski

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Original language	English
Title of host publication	Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing
Publisher	Association for Computational Linguistics
Publication date	1 Nov 2021
Publication status	Published - 1 Nov 2021
Event	Workshop on Simple and Efficient Natural Language Processing - VIRTUAL Duration: 10 Nov 2021 → … Conference number: 2

Workshop

Workshop	Workshop on Simple and Efficient Natural Language Processing
Number	2
City	VIRTUAL
Period	10/11/2021 → …

Access to Document

https://aclanthology.org/2021.sustainlp-1.12.pdfLicence: CC BY

Cite this

@inproceedings{44cde6d2f4054ee896116dc6ad6e124a,

title = "Hyperparameter Power Impact in Transformer Language Model Training",

author = "\{Puvis de Chavannes\}, \{Lucas H{\o}yberg\} and Kongsbak, \{Mads Guldborg Kjeldgaard\} and Timmie Rantzau and Leon Derczynski",

year = "2021",

month = nov,

day = "1",

language = "English",

booktitle = "Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing",

publisher = "Association for Computational Linguistics",

address = "United States",

note = "Workshop on Simple and Efficient Natural Language Processing ; Conference date: 10-11-2021",

}

Puvis de Chavannes, LH, Kongsbak, MGK, Rantzau, T & Derczynski, L 2021, Hyperparameter Power Impact in Transformer Language Model Training. in Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing. Association for Computational Linguistics, Workshop on Simple and Efficient Natural Language Processing, VIRTUAL, 10/11/2021. <https://aclanthology.org/2021.sustainlp-1.12.pdf>

Hyperparameter Power Impact in Transformer Language Model Training. / Puvis de Chavannes, Lucas Høyberg ; Kongsbak, Mads Guldborg Kjeldgaard; Rantzau, Timmie et al.
Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing. Association for Computational Linguistics, 2021.

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

TY - GEN

T1 - Hyperparameter Power Impact in Transformer Language Model Training

AU - Puvis de Chavannes, Lucas Høyberg

AU - Kongsbak, Mads Guldborg Kjeldgaard

AU - Rantzau, Timmie

AU - Derczynski, Leon

N1 - Conference code: 2

PY - 2021/11/1

Y1 - 2021/11/1

M3 - Article in proceedings

BT - Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing

PB - Association for Computational Linguistics

T2 - Workshop on Simple and Efficient Natural Language Processing

Y2 - 10 November 2021

ER -