Findings of the VarDial Evaluation Campaign 2025: The NorSID Shared Task on Norwegian Slot, Intent and Dialect Identification

Yves Scherrer; Rob van der Goot; Petter Mæhlum

Findings of the VarDial Evaluation Campaign 2025: The NorSID Shared Task on Norwegian Slot, Intent and Dialect Identification

Yves Scherrer, Rob van der Goot, Petter Mæhlum

University of Oslo

Publikation: Konference artikel i Proceeding eller bog/rapport kapitel › Konferencebidrag i proceedings › Forskning › peer review

Abstract

The VarDial Evaluation Campaign 2025 was organized as part of the twelfth workshop on Natural Language Processing for Similar Languages, Varieties and Dialects (VarDial), colocated with COLING 2025. It consisted of one shared task with three subtasks: intent detection, slot filling and dialect identification for Norwegian dialects. This report presents the results of this shared task. Four participating teams have submitted systems with very high performance (> 97% accuracy) for intent detection, whereas slot detection and dialect identification showed to be much more challenging, with respectively span-F1 scores up to 89%, and weighted dialect F1 scores of 84%.

Originalsprog	Engelsk
Titel	Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects
Redaktører	Yves Scherrer, Tommi Jauhiainen, Nikola Ljubešić, Preslav Nakov, Jorg Tiedemann, Marcos Zampieri
Antal sider	8
Udgivelsessted	Abu Dhabi, UAE
Forlag	Association for Computational Linguistics
Publikationsdato	1 jan. 2025
Sider	1-8
Status	Udgivet - 1 jan. 2025

Adgang til dokumentet

Citationsformater

Scherrer, Y., van der Goot, R., & Mæhlum, P. (2025). Findings of the VarDial Evaluation Campaign 2025: The NorSID Shared Task on Norwegian Slot, Intent and Dialect Identification. I Y. Scherrer, T. Jauhiainen, N. Ljubešić, P. Nakov, J. Tiedemann, & M. Zampieri (red.), Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects (s. 1-8). Association for Computational Linguistics. https://aclanthology.org/2025.vardial-1.1/

Scherrer, Yves ; van der Goot, Rob ; Mæhlum, Petter. / Findings of the VarDial Evaluation Campaign 2025: The NorSID Shared Task on Norwegian Slot, Intent and Dialect Identification. Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects. red. / Yves Scherrer ; Tommi Jauhiainen ; Nikola Ljubešić ; Preslav Nakov ; Jorg Tiedemann ; Marcos Zampieri. Abu Dhabi, UAE : Association for Computational Linguistics, 2025. s. 1-8

@inproceedings{1bb2d88d67024e61af01e99bb610ac7a,

title = "Findings of the VarDial Evaluation Campaign 2025: The NorSID Shared Task on Norwegian Slot, Intent and Dialect Identification",

abstract = "The VarDial Evaluation Campaign 2025 was organized as part of the twelfth workshop on Natural Language Processing for Similar Languages, Varieties and Dialects (VarDial), colocated with COLING 2025. It consisted of one shared task with three subtasks: intent detection, slot filling and dialect identification for Norwegian dialects. This report presents the results of this shared task. Four participating teams have submitted systems with very high performance (> 97\% accuracy) for intent detection, whereas slot detection and dialect identification showed to be much more challenging, with respectively span-F1 scores up to 89\%, and weighted dialect F1 scores of 84\%.",

keywords = "dialect identification, intent detection, slot filling, Norwegian dialects, VarDial Evaluation Campaign 2025",

author = "Yves Scherrer and \{van der Goot\}, Rob and Petter M{\ae}hlum",

year = "2025",

month = jan,

day = "1",

language = "English",

pages = "1--8",

editor = "Yves Scherrer and Tommi Jauhiainen and Nikola Ljube{\v s}i{\'c} and Preslav Nakov and Jorg Tiedemann and Marcos Zampieri",

booktitle = "Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects",

publisher = "Association for Computational Linguistics",

address = "United States",

}

Scherrer, Y, van der Goot, R & Mæhlum, P 2025, Findings of the VarDial Evaluation Campaign 2025: The NorSID Shared Task on Norwegian Slot, Intent and Dialect Identification. i Y Scherrer, T Jauhiainen, N Ljubešić, P Nakov, J Tiedemann & M Zampieri (red), Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects. Association for Computational Linguistics, Abu Dhabi, UAE, s. 1-8. <https://aclanthology.org/2025.vardial-1.1/>

Findings of the VarDial Evaluation Campaign 2025: The NorSID Shared Task on Norwegian Slot, Intent and Dialect Identification. / Scherrer, Yves; van der Goot, Rob; Mæhlum, Petter.
Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects. red. / Yves Scherrer; Tommi Jauhiainen; Nikola Ljubešić; Preslav Nakov; Jorg Tiedemann; Marcos Zampieri. Abu Dhabi, UAE: Association for Computational Linguistics, 2025. s. 1-8.

Publikation: Konference artikel i Proceeding eller bog/rapport kapitel › Konferencebidrag i proceedings › Forskning › peer review

TY - GEN

T1 - Findings of the VarDial Evaluation Campaign 2025: The NorSID Shared Task on Norwegian Slot, Intent and Dialect Identification

AU - Scherrer, Yves

AU - van der Goot, Rob

AU - Mæhlum, Petter

PY - 2025/1/1

Y1 - 2025/1/1

N2 - The VarDial Evaluation Campaign 2025 was organized as part of the twelfth workshop on Natural Language Processing for Similar Languages, Varieties and Dialects (VarDial), colocated with COLING 2025. It consisted of one shared task with three subtasks: intent detection, slot filling and dialect identification for Norwegian dialects. This report presents the results of this shared task. Four participating teams have submitted systems with very high performance (> 97% accuracy) for intent detection, whereas slot detection and dialect identification showed to be much more challenging, with respectively span-F1 scores up to 89%, and weighted dialect F1 scores of 84%.

AB - The VarDial Evaluation Campaign 2025 was organized as part of the twelfth workshop on Natural Language Processing for Similar Languages, Varieties and Dialects (VarDial), colocated with COLING 2025. It consisted of one shared task with three subtasks: intent detection, slot filling and dialect identification for Norwegian dialects. This report presents the results of this shared task. Four participating teams have submitted systems with very high performance (> 97% accuracy) for intent detection, whereas slot detection and dialect identification showed to be much more challenging, with respectively span-F1 scores up to 89%, and weighted dialect F1 scores of 84%.

KW - dialect identification

KW - intent detection

KW - slot filling

KW - Norwegian dialects

KW - VarDial Evaluation Campaign 2025

M3 - Article in proceedings

SP - 1

EP - 8

BT - Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects

A2 - Scherrer, Yves

A2 - Jauhiainen, Tommi

A2 - Ljubešić, Nikola

A2 - Nakov, Preslav

A2 - Tiedemann, Jorg

A2 - Zampieri, Marcos

PB - Association for Computational Linguistics

CY - Abu Dhabi, UAE

ER -

Scherrer Y, van der Goot R, Mæhlum P. Findings of the VarDial Evaluation Campaign 2025: The NorSID Shared Task on Norwegian Slot, Intent and Dialect Identification. I Scherrer Y, Jauhiainen T, Ljubešić N, Nakov P, Tiedemann J, Zampieri M, red., Proceedings of the 12th Workshop on NLP for Similar Languages, Varieties and Dialects. Abu Dhabi, UAE: Association for Computational Linguistics. 2025. s. 1-8

Findings of the VarDial Evaluation Campaign 2025: The NorSID Shared Task on Norwegian Slot, Intent and Dialect Identification

Abstract

Adgang til dokumentet

Fingeraftryk

Citationsformater