Cross-lingual tagger evaluation without test data

Zeljko Agic; Barbara Plank; Anders Søgaard

Cross-lingual tagger evaluation without test data

Zeljko Agic, Barbara Plank, Anders Søgaard

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Abstract

We address the challenge of cross-lingual POS tagger evaluation in absence of manually annotated test data. We put forth and evaluate two dictionary-based metrics. On the tasks of accuracy prediction and system ranking, we reveal that these metrics are reliable enough to approximate test set-based evaluation, and at the same time lean enough to support assessment for truly low-resource languages.

Original language	English
Title of host publication	Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics
Publisher	Association for Computational Linguistics
Publication date	2017
Pages	248-253
ISBN (Electronic)	978-1-945626-35-7
Publication status	Published - 2017
Event	The 15th Conference of the European Chapter of the Association for Computational Linguistics - Valencia, Spain Duration: 3 Apr 2017 → 7 Apr 2017 http://eacl2017.org/

Conference

Conference	The 15th Conference of the European Chapter of the Association for Computational Linguistics
Country/Territory	Spain
City	Valencia
Period	03/04/2017 → 07/04/2017
Internet address	http://eacl2017.org/

Keywords

Cross-lingual POS tagging
Dictionary-based evaluation metrics
Accuracy prediction
System ranking
Low-resource languages assessment

Access to Document

agic2017-cross.preprintAccepted author manuscript, 782 KB

http://aclweb.org/anthology/E/E17/E17-2040.pdf

Cite this

@inproceedings{a5d9cfb2048242da8ef2f5260f23e618,

title = "Cross-lingual tagger evaluation without test data",

abstract = "We address the challenge of cross-lingual POS tagger evaluation in absence of manually annotated test data. We put forth and evaluate two dictionary-based metrics. On the tasks of accuracy prediction and system ranking, we reveal that these metrics are reliable enough to approximate test set-based evaluation, and at the same time lean enough to support assessment for truly low-resource languages.",

keywords = "Cross-lingual POS tagging, Dictionary-based evaluation metrics, Accuracy prediction, System ranking, Low-resource languages assessment",

author = "Zeljko Agic and Barbara Plank and Anders S{\o}gaard",

year = "2017",

language = "English",

pages = "248--253",

booktitle = "Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics",

address = "United States",

note = "The 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017 ; Conference date: 03-04-2017 Through 07-04-2017",

url = "http://eacl2017.org/",

}

Agic, Z, Plank, B & Søgaard, A 2017, Cross-lingual tagger evaluation without test data. in Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, pp. 248-253, The 15th Conference of the European Chapter of the Association for Computational Linguistics, Valencia, Spain, 03/04/2017. <http://aclweb.org/anthology/E/E17/E17-2040.pdf>

Cross-lingual tagger evaluation without test data. / Agic, Zeljko; Plank, Barbara; Søgaard, Anders.
Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 2017. p. 248-253.

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

TY - GEN

T1 - Cross-lingual tagger evaluation without test data

AU - Agic, Zeljko

AU - Plank, Barbara

AU - Søgaard, Anders

PY - 2017

Y1 - 2017

N2 - We address the challenge of cross-lingual POS tagger evaluation in absence of manually annotated test data. We put forth and evaluate two dictionary-based metrics. On the tasks of accuracy prediction and system ranking, we reveal that these metrics are reliable enough to approximate test set-based evaluation, and at the same time lean enough to support assessment for truly low-resource languages.

AB - We address the challenge of cross-lingual POS tagger evaluation in absence of manually annotated test data. We put forth and evaluate two dictionary-based metrics. On the tasks of accuracy prediction and system ranking, we reveal that these metrics are reliable enough to approximate test set-based evaluation, and at the same time lean enough to support assessment for truly low-resource languages.

KW - Cross-lingual POS tagging

KW - Dictionary-based evaluation metrics

KW - Accuracy prediction

KW - System ranking

KW - Low-resource languages assessment

M3 - Article in proceedings

SP - 248

EP - 253

BT - Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics

PB - Association for Computational Linguistics

T2 - The 15th Conference of the European Chapter of the Association for Computational Linguistics

Y2 - 3 April 2017 through 7 April 2017

ER -

Cross-lingual tagger evaluation without test data

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this