Do Syntactic Categories Help in Developmentally Motivated Curriculum Learning for Language Models?

Arzu Burcu Güven; Anna Rogers; Rob Van Der Goot

doi:10.18653/v1/2025.babylm-main.22

Do Syntactic Categories Help in Developmentally Motivated Curriculum Learning for Language Models?

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Abstract

We examine the syntactic properties of BabyLM corpus, and age-groups within CHILDES. While we find that CHILDES does not exhibit strong syntactic differentiation by age, we show that the syntactic knowledge about the training data can be helpful in interpreting model performance on linguistic tasks. For curriculum learning, we explore developmental and several alternative cognitively inspired curriculum approaches. We find that some curricula help with reading tasks, but the main performance improvement come from using the subset of syntactically categorizable data, rather than the full noisy corpus.

Original language	English
Title of host publication	Proceedings of the First BabyLM Workshop
Editors	Lucas Charpentier, Leshem Choshen, Ryan Cotterell, Mustafa Omer Gul, Michael Y. Hu, Jing Liu, Jaap Jumelet, Tal Linzen, Aaron Mueller, Candace Ross, Raj Sanjay Shah, Alex Warstadt, Ethan Gotlieb Wilcox, Adina Williams
Number of pages	13
Place of Publication	Suzhou, China
Publisher	Association for Computational Linguistics
Publication date	1 Nov 2025
Pages	288-300
ISBN (Print)	TODO
DOIs	https://doi.org/10.18653/v1/2025.babylm-main.22
Publication status	Published - 1 Nov 2025
Event	Workshop on BabyLM: Accelerating Language Modeling Research with Cognitively Plausible Data - Suzhou, China Duration: 5 Nov 2025 → 9 Nov 2025 Conference number: 1 https://aclanthology.org/events/babylm-2025/?utm

Conference

Conference	Workshop on BabyLM
Number	1
Country/Territory	China
City	Suzhou
Period	05/11/2025 → 09/11/2025
Internet address	https://aclanthology.org/events/babylm-2025/?utm

Keywords

Syntactic development
CHILDES corpus
BabyLM corpus
Curriculum learning
Language model evaluation

Access to Document

10.18653/v1/2025.babylm-main.22Licence: CC BY

Cite this

Güven, A. B., Rogers, A., & Goot, R. V. D. (2025). Do Syntactic Categories Help in Developmentally Motivated Curriculum Learning for Language Models? In L. Charpentier, L. Choshen, R. Cotterell, M. O. Gul, M. Y. Hu, J. Liu, J. Jumelet, T. Linzen, A. Mueller, C. Ross, R. S. Shah, A. Warstadt, E. G. Wilcox, & A. Williams (Eds.), Proceedings of the First BabyLM Workshop (pp. 288-300). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.babylm-main.22

Güven, Arzu Burcu ; Rogers, Anna ; Goot, Rob Van Der. / Do Syntactic Categories Help in Developmentally Motivated Curriculum Learning for Language Models?. Proceedings of the First BabyLM Workshop. editor / Lucas Charpentier ; Leshem Choshen ; Ryan Cotterell ; Mustafa Omer Gul ; Michael Y. Hu ; Jing Liu ; Jaap Jumelet ; Tal Linzen ; Aaron Mueller ; Candace Ross ; Raj Sanjay Shah ; Alex Warstadt ; Ethan Gotlieb Wilcox ; Adina Williams. Suzhou, China : Association for Computational Linguistics, 2025. pp. 288-300

@inproceedings{2683ff2133d24a8da409d9f233b193e5,

title = "Do Syntactic Categories Help in Developmentally Motivated Curriculum Learning for Language Models?",

abstract = "We examine the syntactic properties of BabyLM corpus, and age-groups within CHILDES. While we find that CHILDES does not exhibit strong syntactic differentiation by age, we show that the syntactic knowledge about the training data can be helpful in interpreting model performance on linguistic tasks. For curriculum learning, we explore developmental and several alternative cognitively inspired curriculum approaches. We find that some curricula help with reading tasks, but the main performance improvement come from using the subset of syntactically categorizable data, rather than the full noisy corpus.",

keywords = "Syntactic development, CHILDES corpus, BabyLM corpus, Curriculum learning, Language model evaluation",

author = "G{\"u}ven, \{Arzu Burcu\} and Anna Rogers and Goot, \{Rob Van Der\}",

year = "2025",

month = nov,

day = "1",

doi = "10.18653/v1/2025.babylm-main.22",

language = "English",

isbn = "TODO",

pages = "288--300",

editor = "Lucas Charpentier and Leshem Choshen and Ryan Cotterell and Gul, \{Mustafa Omer\} and Hu, \{Michael Y.\} and Jing Liu and Jaap Jumelet and Tal Linzen and Aaron Mueller and Candace Ross and Shah, \{Raj Sanjay\} and Alex Warstadt and Wilcox, \{Ethan Gotlieb\} and Adina Williams",

booktitle = "Proceedings of the First BabyLM Workshop",

publisher = "Association for Computational Linguistics",

address = "United States",

note = "Workshop on BabyLM : Accelerating Language Modeling Research with Cognitively Plausible Data, ACL ; Conference date: 05-11-2025 Through 09-11-2025",

url = "https://aclanthology.org/events/babylm-2025/?utm",

}

Güven, AB , Rogers, A & Goot, RVD 2025, Do Syntactic Categories Help in Developmentally Motivated Curriculum Learning for Language Models? in L Charpentier, L Choshen, R Cotterell, MO Gul, MY Hu, J Liu, J Jumelet, T Linzen, A Mueller, C Ross, RS Shah, A Warstadt, EG Wilcox & A Williams (eds), Proceedings of the First BabyLM Workshop. Association for Computational Linguistics, Suzhou, China, pp. 288-300, Workshop on BabyLM, Suzhou, China, 05/11/2025. https://doi.org/10.18653/v1/2025.babylm-main.22

Do Syntactic Categories Help in Developmentally Motivated Curriculum Learning for Language Models? / Güven, Arzu Burcu ; Rogers, Anna ; Goot, Rob Van Der.
Proceedings of the First BabyLM Workshop. ed. / Lucas Charpentier; Leshem Choshen; Ryan Cotterell; Mustafa Omer Gul; Michael Y. Hu; Jing Liu; Jaap Jumelet; Tal Linzen; Aaron Mueller; Candace Ross; Raj Sanjay Shah; Alex Warstadt; Ethan Gotlieb Wilcox; Adina Williams. Suzhou, China: Association for Computational Linguistics, 2025. p. 288-300.

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review