Experimental Standards for Deep Learning Research: A Natural Language Processing Perspective

Dennis Thomas Ulmer; Elisa Bassignana; Max Müller-Eberstein; Daniel Varab; Mike Zhang; Christian Hardmeier; Barbara Plank

Experimental Standards for Deep Learning Research: A Natural Language Processing Perspective

Dennis Thomas Ulmer, Elisa Bassignana, Max Müller-Eberstein, Daniel Varab, Mike Zhang, Christian Hardmeier, Barbara Plank

Research output: Contribution to conference - NOT published in proceeding or journal › Paper › Research › peer-review

Abstract

The field of Deep Learning (DL) has undergone explosive growth during the last decade, with a substantial impact on Natural Language Processing (NLP) as well. Yet, as with other fields employing DL techniques, there has been a lack of common experimental standards compared to more established disciplines. Starting from fundamental scientific principles, we distill ongoing discussions on experimental standards in DL into a single, widely-applicable methodology. Following these best practices is crucial to strengthening experimental evidence, improve reproducibility and enable scientific progress. These standards are further collected in a public repository to help them transparently adapt to future needs.

Original language	English
Publication date	29 Apr 2022
Publication status	Published - 29 Apr 2022
Event	ML Evaluation Standards Workshop at ICLR 2022 - Duration: 29 Apr 2022 → …

Conference

Conference	ML Evaluation Standards Workshop at ICLR 2022
Period	29/04/2022 → …

Keywords

Deep Learning
Natural Language Processing
Experimental Standards
Reproducibility
Scientific Methodology

Access to Document

2204.06251Final published version, 544 KB

Cite this

@conference{fc23a8ce731e4697aad6c6f8388c5a44,

title = "Experimental Standards for Deep Learning Research: A Natural Language Processing Perspective",

abstract = "The field of Deep Learning (DL) has undergone explosive growth during the last decade, with a substantial impact on Natural Language Processing (NLP) as well. Yet, as with other fields employing DL techniques, there has been a lack of common experimental standards compared to more established disciplines. Starting from fundamental scientific principles, we distill ongoing discussions on experimental standards in DL into a single, widely-applicable methodology. Following these best practices is crucial to strengthening experimental evidence, improve reproducibility and enable scientific progress. These standards are further collected in a public repository to help them transparently adapt to future needs.",

keywords = "Deep Learning, Natural Language Processing, Experimental Standards, Reproducibility, Scientific Methodology, Deep Learning, Natural Language Processing, Experimental Standards, Reproducibility, Scientific Methodology",

author = "Ulmer, {Dennis Thomas} and Elisa Bassignana and Max M{\"u}ller-Eberstein and Daniel Varab and Mike Zhang and Christian Hardmeier and Barbara Plank",

year = "2022",

month = apr,

day = "29",

language = "English",

note = "ML Evaluation Standards Workshop at ICLR 2022 ; Conference date: 29-04-2022",

}

TY - CONF

T1 - Experimental Standards for Deep Learning Research: A Natural Language Processing Perspective

AU - Ulmer, Dennis Thomas

AU - Bassignana, Elisa

AU - Müller-Eberstein, Max

AU - Varab, Daniel

AU - Zhang, Mike

AU - Hardmeier, Christian

AU - Plank, Barbara

PY - 2022/4/29

Y1 - 2022/4/29

N2 - The field of Deep Learning (DL) has undergone explosive growth during the last decade, with a substantial impact on Natural Language Processing (NLP) as well. Yet, as with other fields employing DL techniques, there has been a lack of common experimental standards compared to more established disciplines. Starting from fundamental scientific principles, we distill ongoing discussions on experimental standards in DL into a single, widely-applicable methodology. Following these best practices is crucial to strengthening experimental evidence, improve reproducibility and enable scientific progress. These standards are further collected in a public repository to help them transparently adapt to future needs.

AB - The field of Deep Learning (DL) has undergone explosive growth during the last decade, with a substantial impact on Natural Language Processing (NLP) as well. Yet, as with other fields employing DL techniques, there has been a lack of common experimental standards compared to more established disciplines. Starting from fundamental scientific principles, we distill ongoing discussions on experimental standards in DL into a single, widely-applicable methodology. Following these best practices is crucial to strengthening experimental evidence, improve reproducibility and enable scientific progress. These standards are further collected in a public repository to help them transparently adapt to future needs.

KW - Deep Learning

KW - Natural Language Processing

KW - Experimental Standards

KW - Reproducibility

KW - Scientific Methodology

KW - Deep Learning

KW - Natural Language Processing

KW - Experimental Standards

KW - Reproducibility

KW - Scientific Methodology

UR - https://arxiv.org/pdf/2204.06251.pdf

M3 - Paper

T2 - ML Evaluation Standards Workshop at ICLR 2022

Y2 - 29 April 2022

ER -

Experimental Standards for Deep Learning Research: A Natural Language Processing Perspective

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this