How to age BERT Well: Continuous Training for Historical Language Adaptation

Anika Harju; Rob van der Goot

How to age BERT Well: Continuous Training for Historical Language Adaptation

Anika Harju
, Rob van der Goot

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Original language	English
Title of host publication	Proceedings of the First Workshop on Language Models for Low-Resource Languages
Editors	Hansi Hettiarachchi, Tharindu Ranasinghe, Paul Rayson, Ruslan Mitkov, Mohamed Gaber, Damith Premasiri, Fiona Anting Tan, Lasitha Uyangodage
Number of pages	10
Place of Publication	Abu Dhabi, United Arab Emirates
Publisher	Association for Computational Linguistics
Publication date	1 Jan 2025
Pages	258-267
Publication status	Published - 1 Jan 2025
Event	Workshop on Language Models for Low-Resource Languages - Abu Dhabi National Exhibition Centre, Abu Dhabi, United Arab Emirates Duration: 19 Jan 2025 → 20 Jan 2025 Conference number: 1 http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=181589 https://loreslm.github.io/program?utm

Workshop

Workshop	Workshop on Language Models for Low-Resource Languages
Number	1
Location	Abu Dhabi National Exhibition Centre
Country/Territory	United Arab Emirates
City	Abu Dhabi
Period	19/01/2025 → 20/01/2025
Internet address	http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=181589 https://loreslm.github.io/program?utm

Access to Document

https://aclanthology.org/2025.loreslm-1.21.pdfLicence: CC BY
https://aclanthology.org/2025.loreslm-1.21/Licence: CC BY

Cite this

Harju, A., & van der Goot, R. (2025). How to age BERT Well: Continuous Training for Historical Language Adaptation. In H. Hettiarachchi, T. Ranasinghe, P. Rayson, R. Mitkov, M. Gaber, D. Premasiri, F. A. Tan, & L. Uyangodage (Eds.), Proceedings of the First Workshop on Language Models for Low-Resource Languages (pp. 258-267). Association for Computational Linguistics. https://aclanthology.org/2025.loreslm-1.21.pdf

Harju, Anika ; van der Goot, Rob. / How to age BERT Well: Continuous Training for Historical Language Adaptation. Proceedings of the First Workshop on Language Models for Low-Resource Languages. editor / Hansi Hettiarachchi ; Tharindu Ranasinghe ; Paul Rayson ; Ruslan Mitkov ; Mohamed Gaber ; Damith Premasiri ; Fiona Anting Tan ; Lasitha Uyangodage. Abu Dhabi, United Arab Emirates : Association for Computational Linguistics, 2025. pp. 258-267

@inproceedings{1e22a358282a42d9a0c54a4d1115f247,

title = "How to age BERT Well: Continuous Training for Historical Language Adaptation",

author = "Anika Harju and \{van der Goot\}, Rob",

year = "2025",

month = jan,

day = "1",

language = "English",

pages = "258--267",

editor = "Hansi Hettiarachchi and Tharindu Ranasinghe and Paul Rayson and Ruslan Mitkov and Mohamed Gaber and Damith Premasiri and Tan, \{Fiona Anting\} and Lasitha Uyangodage",

booktitle = "Proceedings of the First Workshop on Language Models for Low-Resource Languages",

publisher = "Association for Computational Linguistics",

address = "United States",

note = "Workshop on Language Models for Low-Resource Languages, LoResLM ; Conference date: 19-01-2025 Through 20-01-2025",

url = "http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=181589, https://loreslm.github.io/program?utm",

}

Harju, A & van der Goot, R 2025, How to age BERT Well: Continuous Training for Historical Language Adaptation. in H Hettiarachchi, T Ranasinghe, P Rayson, R Mitkov, M Gaber, D Premasiri, FA Tan & L Uyangodage (eds), Proceedings of the First Workshop on Language Models for Low-Resource Languages. Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, pp. 258-267, Workshop on Language Models for Low-Resource Languages, Abu Dhabi, United Arab Emirates, 19/01/2025. <https://aclanthology.org/2025.loreslm-1.21.pdf>

How to age BERT Well: Continuous Training for Historical Language Adaptation. / Harju, Anika; van der Goot, Rob.
Proceedings of the First Workshop on Language Models for Low-Resource Languages. ed. / Hansi Hettiarachchi; Tharindu Ranasinghe; Paul Rayson; Ruslan Mitkov; Mohamed Gaber; Damith Premasiri; Fiona Anting Tan; Lasitha Uyangodage. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, 2025. p. 258-267.

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

TY - GEN

T1 - How to age BERT Well: Continuous Training for Historical Language Adaptation

AU - Harju, Anika

AU - van der Goot, Rob

N1 - Conference code: 1

PY - 2025/1/1

Y1 - 2025/1/1

M3 - Article in proceedings

SP - 258

EP - 267

BT - Proceedings of the First Workshop on Language Models for Low-Resource Languages

A2 - Hettiarachchi, Hansi

A2 - Ranasinghe, Tharindu

A2 - Rayson, Paul

A2 - Mitkov, Ruslan

A2 - Gaber, Mohamed

A2 - Premasiri, Damith

A2 - Tan, Fiona Anting

A2 - Uyangodage, Lasitha

PB - Association for Computational Linguistics

CY - Abu Dhabi, United Arab Emirates

T2 - Workshop on Language Models for Low-Resource Languages

Y2 - 19 January 2025 through 20 January 2025

ER -

Harju A, van der Goot R. How to age BERT Well: Continuous Training for Historical Language Adaptation. In Hettiarachchi H, Ranasinghe T, Rayson P, Mitkov R, Gaber M, Premasiri D, Tan FA, Uyangodage L, editors, Proceedings of the First Workshop on Language Models for Low-Resource Languages. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics. 2025. p. 258-267