Neural Unsupervised Domain Adaptation in NLP—A Survey

Alan Ramponi; Barbara Plank

Neural Unsupervised Domain Adaptation in NLP—A Survey

Alan Ramponi
, Barbara Plank

NLPnorth

University of Trento

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Abstract

Deep neural networks excel at learning from labeled data and achieve state-of-the-art results on a wide array of Natural Language Processing tasks. In contrast, learning from unlabeled data, especially under domain shift, remains a challenge.
Motivated by the latest advances, in this survey we review neural unsupervised domain adaptation techniques which do not require labeled target domain data. This is a more challenging yet a more widely applicable setup. We outline methods, from early traditional non-neural methods to pre-trained model transfer. We also revisit the notion of domain, and we uncover a bias in the type of Natural Language Processing tasks which received most attention. Lastly, we outline future directions, particularly the broader need for out-of-distribution generalization of future NLP.

Original language	English
Title of host publication	The 28th International Conference on Computational Linguistics
Publisher	Association for Computational Linguistics
Publication date	Dec 2020
Publication status	Published - Dec 2020
Event	International Conference on Computational Linguistics - Barcelona, Spain Duration: 8 Dec 2020 → 13 Dec 2020 Conference number: 28th

Conference

Conference	International Conference on Computational Linguistics
Number	28th
Country/Territory	Spain
City	Barcelona
Period	08/12/2020 → 13/12/2020

Keywords

Unsupervised Domain Adaptation
Neural Networks
Natural Language Processing
Domain Shift
Out-of-Distribution Generalization

Access to Document

COLING2020__Neural_Unsupervised_Domain_Adaptation_in_NLP___A_SurveyAccepted author manuscript, 430 KB

Cite this

@inproceedings{40bf65a217fe472db595ebf92fc54ba7,

title = "Neural Unsupervised Domain Adaptation in NLP—A Survey",

abstract = "Deep neural networks excel at learning from labeled data and achieve state-of-the-art results on a wide array of Natural Language Processing tasks. In contrast, learning from unlabeled data, especially under domain shift, remains a challenge. Motivated by the latest advances, in this survey we review neural unsupervised domain adaptation techniques which do not require labeled target domain data. This is a more challenging yet a more widely applicable setup. We outline methods, from early traditional non-neural methods to pre-trained model transfer. We also revisit the notion of domain, and we uncover a bias in the type of Natural Language Processing tasks which received most attention. Lastly, we outline future directions, particularly the broader need for out-of-distribution generalization of future NLP.",

keywords = "Unsupervised Domain Adaptation, Neural Networks, Natural Language Processing, Domain Shift, Out-of-Distribution Generalization, Unsupervised Domain Adaptation, Neural Networks, Natural Language Processing, Domain Shift, Out-of-Distribution Generalization",

author = "Alan Ramponi and Barbara Plank",

year = "2020",

month = dec,

language = "English",

booktitle = "The 28th International Conference on Computational Linguistics",

publisher = "Association for Computational Linguistics",

address = "United States",

note = "International Conference on Computational Linguistics ; Conference date: 08-12-2020 Through 13-12-2020",

}

TY - GEN

T1 - Neural Unsupervised Domain Adaptation in NLP—A Survey

AU - Ramponi, Alan

AU - Plank, Barbara

N1 - Conference code: 28th

PY - 2020/12

Y1 - 2020/12

N2 - Deep neural networks excel at learning from labeled data and achieve state-of-the-art results on a wide array of Natural Language Processing tasks. In contrast, learning from unlabeled data, especially under domain shift, remains a challenge. Motivated by the latest advances, in this survey we review neural unsupervised domain adaptation techniques which do not require labeled target domain data. This is a more challenging yet a more widely applicable setup. We outline methods, from early traditional non-neural methods to pre-trained model transfer. We also revisit the notion of domain, and we uncover a bias in the type of Natural Language Processing tasks which received most attention. Lastly, we outline future directions, particularly the broader need for out-of-distribution generalization of future NLP.

AB - Deep neural networks excel at learning from labeled data and achieve state-of-the-art results on a wide array of Natural Language Processing tasks. In contrast, learning from unlabeled data, especially under domain shift, remains a challenge. Motivated by the latest advances, in this survey we review neural unsupervised domain adaptation techniques which do not require labeled target domain data. This is a more challenging yet a more widely applicable setup. We outline methods, from early traditional non-neural methods to pre-trained model transfer. We also revisit the notion of domain, and we uncover a bias in the type of Natural Language Processing tasks which received most attention. Lastly, we outline future directions, particularly the broader need for out-of-distribution generalization of future NLP.

KW - Unsupervised Domain Adaptation

KW - Neural Networks

KW - Natural Language Processing

KW - Domain Shift

KW - Out-of-Distribution Generalization

KW - Unsupervised Domain Adaptation

KW - Neural Networks

KW - Natural Language Processing

KW - Domain Shift

KW - Out-of-Distribution Generalization

UR - https://www.aclweb.org/anthology/2020.coling-main.603.pdf

M3 - Article in proceedings

BT - The 28th International Conference on Computational Linguistics

PB - Association for Computational Linguistics

T2 - International Conference on Computational Linguistics

Y2 - 8 December 2020 through 13 December 2020

ER -

Neural Unsupervised Domain Adaptation in NLP—A Survey

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this