A Case for Soft Loss Functions

Alexandra Uma; Tommaso Fornaciari; Dirk Hovy; Silviu Paun; Barbara Plank; Massimo Poesio

A Case for Soft Loss Functions

Alexandra Uma
, Tommaso Fornaciari
, Dirk Hovy
, Silviu Paun
, Barbara Plank
, Massimo Poesio

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Abstract

Recently, Peterson et al. provided evidence of the benefits of using probabilistic soft labels generated from crowd annotations for training a computer vision model, showing that us- ing such labels maximizes performance of the models over unseen data. In this paper, we generalize these results by showing that training with soft labels is an effective method for using crowd annotations in several other AI tasks besides the one studied by Peterson et al., and also when their performance is compared with that of state-of-the-art methods for learning from crowdsourced data.

Original language	English
Title of host publication	Proceedings of the eighth AAAI Conference on Human Computation and Crowdsourcing
Publisher	AAAI Press
Publication date	2020
Publication status	Published - 2020
Event	Conference on Human Computation and Crowdsourcing - Online , Hilversum, Netherlands Duration: 25 Oct 2020 → 29 Oct 2020 Conference number: 8th

Conference

Conference	Conference on Human Computation and Crowdsourcing
Number	8th
Location	Online
Country/Territory	Netherlands
City	Hilversum
Period	25/10/2020 → 29/10/2020

Keywords

Probabilistic soft labels
Crowd annotations
Computer vision model
AI tasks
Crowdsourced data

Access to Document

HCOMP_2020_paper_51Accepted author manuscript, 152 KB

Cite this

@inproceedings{5c3b396c485d4208bbf2eeb8b959f13d,

title = "A Case for Soft Loss Functions",

abstract = "Recently, Peterson et al. provided evidence of the benefits of using probabilistic soft labels generated from crowd annotations for training a computer vision model, showing that us- ing such labels maximizes performance of the models over unseen data. In this paper, we generalize these results by showing that training with soft labels is an effective method for using crowd annotations in several other AI tasks besides the one studied by Peterson et al., and also when their performance is compared with that of state-of-the-art methods for learning from crowdsourced data.",

keywords = "Probabilistic soft labels, Crowd annotations, Computer vision model, AI tasks, Crowdsourced data, Probabilistic soft labels, Crowd annotations, Computer vision model, AI tasks, Crowdsourced data",

author = "Alexandra Uma and Tommaso Fornaciari and Dirk Hovy and Silviu Paun and Barbara Plank and Massimo Poesio",

year = "2020",

language = "English",

booktitle = "Proceedings of the eighth AAAI Conference on Human Computation and Crowdsourcing",

publisher = "AAAI Press",

address = "United States",

note = "Conference on Human Computation and Crowdsourcing ; Conference date: 25-10-2020 Through 29-10-2020",

}

TY - GEN

T1 - A Case for Soft Loss Functions

AU - Uma, Alexandra

AU - Fornaciari, Tommaso

AU - Hovy, Dirk

AU - Paun, Silviu

AU - Plank, Barbara

AU - Poesio, Massimo

N1 - Conference code: 8th

PY - 2020

Y1 - 2020

N2 - Recently, Peterson et al. provided evidence of the benefits of using probabilistic soft labels generated from crowd annotations for training a computer vision model, showing that us- ing such labels maximizes performance of the models over unseen data. In this paper, we generalize these results by showing that training with soft labels is an effective method for using crowd annotations in several other AI tasks besides the one studied by Peterson et al., and also when their performance is compared with that of state-of-the-art methods for learning from crowdsourced data.

AB - Recently, Peterson et al. provided evidence of the benefits of using probabilistic soft labels generated from crowd annotations for training a computer vision model, showing that us- ing such labels maximizes performance of the models over unseen data. In this paper, we generalize these results by showing that training with soft labels is an effective method for using crowd annotations in several other AI tasks besides the one studied by Peterson et al., and also when their performance is compared with that of state-of-the-art methods for learning from crowdsourced data.

KW - Probabilistic soft labels

KW - Crowd annotations

KW - Computer vision model

KW - AI tasks

KW - Crowdsourced data

KW - Probabilistic soft labels

KW - Crowd annotations

KW - Computer vision model

KW - AI tasks

KW - Crowdsourced data

M3 - Article in proceedings

BT - Proceedings of the eighth AAAI Conference on Human Computation and Crowdsourcing

PB - AAAI Press

T2 - Conference on Human Computation and Crowdsourcing

Y2 - 25 October 2020 through 29 October 2020

ER -

A Case for Soft Loss Functions

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this