Cross-lingual Multi-task Transfer for Zero-shot Task-oriented Dialog

Rob van der Goot; Marija Stepanovic; Alan Ramponi; Ibrahim Sharaf; Ahmet Üstün; Aizhan  Imankulova; Siti Oryza Khairunnisa; Mamoru Komachi; Barbara Plank

Cross-lingual Multi-task Transfer for Zero-shot Task-oriented Dialog

Rob van der Goot, Marija Stepanovic, Alan Ramponi, Ibrahim Sharaf, Ahmet Üstün, Aizhan Imankulova, Siti Oryza Khairunnisa, Mamoru Komachi, Barbara Plank

Publikation: Konferencebidrag - EJ publiceret i proceeding eller tidsskrift › Konferenceabstrakt til konference › Forskning › peer review

Abstract

Digital assistants are becoming an integral part of everyday life. However, commercial digital assistants are only available for a limited set of languages. Because of this, a vast amount of people can not use these devices in their native tongue.
In this work, we focus on two core tasks within the digital assistant pipeline: intent classification and slot detection. Intent classification recovers the goal of the utterance, whereas slot detection identifies important properties regarding this goal. Besides introducing a novel cross-lingual dataset for these tasks, consisting of 11 languages, we evaluate a variety of models: 1)
multilingually pretrained transformer-based models, 2) we supplement these models with auxiliary tasks to evaluate whether multi-task learning can be beneficial, and 3) annotation transfer with neural machine translation.

Originalsprog	Engelsk
Publikationsdato	25 sep. 2021
Status	Udgivet - 25 sep. 2021
Begivenhed	RESOURCEFUL-2020 : RESOURCEs and representations For Under-resourced Languages and domains - Gothenburg, Gothenburg, Sverige Varighed: 25 nov. 2020 → … https://gu-clasp.github.io/resourceful-2020/

Workshop

Workshop	RESOURCEFUL-2020
Lokation	Gothenburg
Land/Område	Sverige
By	Gothenburg
Periode	25/11/2020 → …
Internetadresse	https://gu-clasp.github.io/resourceful-2020/

Adgang til dokumentet

https://robvanderg.github.io/doc/resourceful2020.1.pdf

Citationsformater

@conference{e5ccae9243444d08a3e775fbd93d60bb,

title = "Cross-lingual Multi-task Transfer for Zero-shot Task-oriented Dialog",

abstract = "Digital assistants are becoming an integral part of everyday life. However, commercial digital assistants are only available for a limited set of languages. Because of this, a vast amount of people can not use these devices in their native tongue.In this work, we focus on two core tasks within the digital assistant pipeline: intent classification and slot detection. Intent classification recovers the goal of the utterance, whereas slot detection identifies important properties regarding this goal. Besides introducing a novel cross-lingual dataset for these tasks, consisting of 11 languages, we evaluate a variety of models: 1)multilingually pretrained transformer-based models, 2) we supplement these models with auxiliary tasks to evaluate whether multi-task learning can be beneficial, and 3) annotation transfer with neural machine translation.",

keywords = "Digital assistants, Intent classification, Slot detection, Cross-lingual dataset, Multilingual transformers",

author = "{van der Goot}, Rob and Marija Stepanovic and Alan Ramponi and Ibrahim Sharaf and Ahmet {\"U}st{\"u}n and Aizhan Imankulova and Khairunnisa, {Siti Oryza} and Mamoru Komachi and Barbara Plank",

year = "2021",

month = sep,

day = "25",

language = "English",

note = "RESOURCEFUL-2020<br/> : RESOURCEs and representations For Under-resourced Languages and domains, RESOURCEFUL ; Conference date: 25-11-2020",

url = "https://gu-clasp.github.io/resourceful-2020/",

}

TY - ABST

T1 - Cross-lingual Multi-task Transfer for Zero-shot Task-oriented Dialog

AU - van der Goot, Rob

AU - Stepanovic, Marija

AU - Ramponi, Alan

AU - Sharaf, Ibrahim

AU - Üstün, Ahmet

AU - Imankulova, Aizhan

AU - Khairunnisa, Siti Oryza

AU - Komachi, Mamoru

AU - Plank, Barbara

PY - 2021/9/25

Y1 - 2021/9/25

N2 - Digital assistants are becoming an integral part of everyday life. However, commercial digital assistants are only available for a limited set of languages. Because of this, a vast amount of people can not use these devices in their native tongue.In this work, we focus on two core tasks within the digital assistant pipeline: intent classification and slot detection. Intent classification recovers the goal of the utterance, whereas slot detection identifies important properties regarding this goal. Besides introducing a novel cross-lingual dataset for these tasks, consisting of 11 languages, we evaluate a variety of models: 1)multilingually pretrained transformer-based models, 2) we supplement these models with auxiliary tasks to evaluate whether multi-task learning can be beneficial, and 3) annotation transfer with neural machine translation.

AB - Digital assistants are becoming an integral part of everyday life. However, commercial digital assistants are only available for a limited set of languages. Because of this, a vast amount of people can not use these devices in their native tongue.In this work, we focus on two core tasks within the digital assistant pipeline: intent classification and slot detection. Intent classification recovers the goal of the utterance, whereas slot detection identifies important properties regarding this goal. Besides introducing a novel cross-lingual dataset for these tasks, consisting of 11 languages, we evaluate a variety of models: 1)multilingually pretrained transformer-based models, 2) we supplement these models with auxiliary tasks to evaluate whether multi-task learning can be beneficial, and 3) annotation transfer with neural machine translation.

KW - Digital assistants

KW - Intent classification

KW - Slot detection

KW - Cross-lingual dataset

KW - Multilingual transformers

M3 - Conference abstract for conference

T2 - RESOURCEFUL-2020<br/>

Y2 - 25 November 2020

ER -

Cross-lingual Multi-task Transfer for Zero-shot Task-oriented Dialog

Abstract

Workshop

Adgang til dokumentet

Fingeraftryk

Citationsformater