A Case for Soft Loss Functions

Alexandra Uma; Tommaso Fornaciari; Dirk Hovy; Silviu Paun; Barbara Plank; Massimo Poesio

A Case for Soft Loss Functions

Alexandra Uma, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, Massimo Poesio

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Abstract

Recently, Peterson et al. provided evidence of the benefits of using probabilistic soft labels generated from crowd annotations for training a computer vision model, showing that us- ing such labels maximizes performance of the models over unseen data. In this paper, we generalize these results by showing that training with soft labels is an effective method for using crowd annotations in several other AI tasks besides the one studied by Peterson et al., and also when their performance is compared with that of state-of-the-art methods for learning from crowdsourced data.

Original language	English
Title of host publication	Proceedings of the eighth AAAI Conference on Human Computation and Crowdsourcing
Publisher	AAAI Press
Publication date	2020
Publication status	Published - 2020

Keywords

Probabilistic soft labels
Crowd annotations
Computer vision model
AI tasks
Crowdsourced data

Access to Document

HCOMP_2020_paper_51Accepted author manuscript, 152 KB

Cite this

@inproceedings{5c3b396c485d4208bbf2eeb8b959f13d,

title = "A Case for Soft Loss Functions",

abstract = "Recently, Peterson et al. provided evidence of the benefits of using probabilistic soft labels generated from crowd annotations for training a computer vision model, showing that us- ing such labels maximizes performance of the models over unseen data. In this paper, we generalize these results by showing that training with soft labels is an effective method for using crowd annotations in several other AI tasks besides the one studied by Peterson et al., and also when their performance is compared with that of state-of-the-art methods for learning from crowdsourced data.",

keywords = "Probabilistic soft labels, Crowd annotations, Computer vision model, AI tasks, Crowdsourced data, Probabilistic soft labels, Crowd annotations, Computer vision model, AI tasks, Crowdsourced data",

author = "Alexandra Uma and Tommaso Fornaciari and Dirk Hovy and Silviu Paun and Barbara Plank and Massimo Poesio",

year = "2020",

language = "English",

booktitle = "Proceedings of the eighth AAAI Conference on Human Computation and Crowdsourcing",

publisher = "AAAI Press",

address = "United States",

}

TY - GEN

T1 - A Case for Soft Loss Functions

AU - Uma, Alexandra

AU - Fornaciari, Tommaso

AU - Hovy, Dirk

AU - Paun, Silviu

AU - Plank, Barbara

AU - Poesio, Massimo

PY - 2020

Y1 - 2020

N2 - Recently, Peterson et al. provided evidence of the benefits of using probabilistic soft labels generated from crowd annotations for training a computer vision model, showing that us- ing such labels maximizes performance of the models over unseen data. In this paper, we generalize these results by showing that training with soft labels is an effective method for using crowd annotations in several other AI tasks besides the one studied by Peterson et al., and also when their performance is compared with that of state-of-the-art methods for learning from crowdsourced data.

AB - Recently, Peterson et al. provided evidence of the benefits of using probabilistic soft labels generated from crowd annotations for training a computer vision model, showing that us- ing such labels maximizes performance of the models over unseen data. In this paper, we generalize these results by showing that training with soft labels is an effective method for using crowd annotations in several other AI tasks besides the one studied by Peterson et al., and also when their performance is compared with that of state-of-the-art methods for learning from crowdsourced data.

KW - Probabilistic soft labels

KW - Crowd annotations

KW - Computer vision model

KW - AI tasks

KW - Crowdsourced data

KW - Probabilistic soft labels

KW - Crowd annotations

KW - Computer vision model

KW - AI tasks

KW - Crowdsourced data

M3 - Article in proceedings

BT - Proceedings of the eighth AAAI Conference on Human Computation and Crowdsourcing

PB - AAAI Press

ER -

A Case for Soft Loss Functions

Abstract

Keywords

Access to Document

Fingerprint

Cite this