Hyperparameter Power Impact in Transformer Language Model Training

Lucas Høyberg  Puvis de Chavannes; Mads Guldborg Kjeldgaard Kongsbak; Timmie Rantzau; Leon Derczynski

Hyperparameter Power Impact in Transformer Language Model Training

Lucas Høyberg Puvis de Chavannes
, Mads Guldborg Kjeldgaard Kongsbak
, Timmie Rantzau
, Leon Derczynski

Publikation: Konference artikel i Proceeding eller bog/rapport kapitel › Konferencebidrag i proceedings › Forskning › peer review

Originalsprog	Engelsk
Titel	Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing
Forlag	Association for Computational Linguistics
Publikationsdato	1 nov. 2021
Status	Udgivet - 1 nov. 2021

Adgang til dokumentet

https://aclanthology.org/2021.sustainlp-1.12.pdfLicens: CC BY

Citationsformater

@inproceedings{44cde6d2f4054ee896116dc6ad6e124a,

title = "Hyperparameter Power Impact in Transformer Language Model Training",

author = "\{Puvis de Chavannes\}, \{Lucas H{\o}yberg\} and Kongsbak, \{Mads Guldborg Kjeldgaard\} and Timmie Rantzau and Leon Derczynski",

year = "2021",

month = nov,

day = "1",

language = "English",

booktitle = "Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing",

publisher = "Association for Computational Linguistics",

address = "United States",

}

Hyperparameter Power Impact in Transformer Language Model Training. / Puvis de Chavannes, Lucas Høyberg ; Kongsbak, Mads Guldborg Kjeldgaard; Rantzau, Timmie et al.
Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing. Association for Computational Linguistics, 2021.

Publikation: Konference artikel i Proceeding eller bog/rapport kapitel › Konferencebidrag i proceedings › Forskning › peer review

TY - GEN

T1 - Hyperparameter Power Impact in Transformer Language Model Training

AU - Puvis de Chavannes, Lucas Høyberg

AU - Kongsbak, Mads Guldborg Kjeldgaard

AU - Rantzau, Timmie

AU - Derczynski, Leon

PY - 2021/11/1

Y1 - 2021/11/1

M3 - Article in proceedings

BT - Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing

PB - Association for Computational Linguistics

ER -