Interactive Learning for Multimedia at Large

Omar Shahbaz Khan; Björn Thór Jónsson; Stevan Rudinac; Jan Zahálka; Hanna Ragnarsdóttir; Þórhildur Þorleiksdóttir; Gylfi Þór Guðmundsson; Laurent Amsaleg; Marcel Worring

doi:10.1007/978-3-030-45439-5_33

Interactive Learning for Multimedia at Large

Omar Shahbaz Khan
, Björn Thór Jónsson
, Stevan Rudinac
, Jan Zahálka
, Hanna Ragnarsdóttir
, Þórhildur Þorleiksdóttir
, Gylfi Þór Guðmundsson
, Laurent Amsaleg
, Marcel Worring

Data-intensive Systems and Applications

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Abstract

Interactive learning has been suggested as a key method for addressing analytic multimedia tasks arising in several domains. Until recently, however, methods to maintain interactive performance at the scale of today’s media collections have not been addressed. We propose an interactive learning approach that builds on and extends the state of the art in user relevance feedback systems and high-dimensional indexing for multimedia. We report on a detailed experimental study using the ImageNet and YFCC100M collections, containing 14 million and 100 million images respectively. The proposed approach outperforms the relevant state-of-the-art approaches in terms of interactive performance, while improving suggestion relevance in some cases. In particular, even on YFCC100M, our approach requires less than 0.3 s per interaction round to generate suggestions, using a single computing core and less than 7 GB of main memory.

Original language	English
Title of host publication	Advances in Information Retrieval : 42nd European Conference on IR Research, ECIR 2020
Number of pages	16
Place of Publication	Lisbon, Portugal
Publisher	Springer
Publication date	Apr 2020
Pages	495-510
ISBN (Print)	978-3-030-45438-8
ISBN (Electronic)	978-3-030-45439-5
DOIs	https://doi.org/10.1007/978-3-030-45439-5_33
Publication status	Published - Apr 2020
Event	European Conference on Information Retrieval - Online , Libson, Portugal Duration: 14 Apr 2020 → 17 Apr 2020 Conference number: 42nd

Conference

Conference	European Conference on Information Retrieval
Number	42nd
Location	Online
Country/Territory	Portugal
City	Libson
Period	14/04/2020 → 17/04/2020

Series	Lecture Notes in Computer Science
Volume	12035
ISSN	0302-9743

Keywords

Large multimedia collections
Interactive multimodal learning
YFCC100M
High-dimensional indexing

Access to Document

10.1007/978-3-030-45439-5_33

Exquisitor_ECIR_submissionAccepted author manuscript, 1.71 MB

Cite this

@inproceedings{6b7f17cf0e654a51ac45b6568c687027,

title = "Interactive Learning for Multimedia at Large",

abstract = "Interactive learning has been suggested as a key method for addressing analytic multimedia tasks arising in several domains. Until recently, however, methods to maintain interactive performance at the scale of today{\textquoteright}s media collections have not been addressed. We propose an interactive learning approach that builds on and extends the state of the art in user relevance feedback systems and high-dimensional indexing for multimedia. We report on a detailed experimental study using the ImageNet and YFCC100M collections, containing 14 million and 100 million images respectively. The proposed approach outperforms the relevant state-of-the-art approaches in terms of interactive performance, while improving suggestion relevance in some cases. In particular, even on YFCC100M, our approach requires less than 0.3 s per interaction round to generate suggestions, using a single computing core and less than 7 GB of main memory.",

keywords = "Large multimedia collections, Interactive multimodal learning, YFCC100M, High-dimensional indexing, Large multimedia collections, Interactive multimodal learning, YFCC100M, High-dimensional indexing",

author = "Khan, \{Omar Shahbaz\} and J{\'o}nsson, \{Bj{\"o}rn Th{\'o}r\} and Stevan Rudinac and Jan Zah{\'a}lka and Hanna Ragnarsd{\'o}ttir and {\TH}{\'o}rhildur {\TH}orleiksd{\'o}ttir and Gu{\dh}mundsson, \{Gylfi {\TH}{\'o}r\} and Laurent Amsaleg and Marcel Worring",

year = "2020",

month = apr,

doi = "10.1007/978-3-030-45439-5\_33",

language = "English",

isbn = "978-3-030-45438-8",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "495--510",

booktitle = "Advances in Information Retrieval",

address = "Germany",

note = " European Conference on Information Retrieval ; Conference date: 14-04-2020 Through 17-04-2020",

}

Khan, OS , Jónsson, BT, Rudinac, S, Zahálka, J, Ragnarsdóttir, H, Þorleiksdóttir, Þ, Guðmundsson, GÞ, Amsaleg, L & Worring, M 2020, Interactive Learning for Multimedia at Large. in Advances in Information Retrieval: 42nd European Conference on IR Research, ECIR 2020. Springer, Lisbon, Portugal, Lecture Notes in Computer Science, vol. 12035, pp. 495-510, European Conference on Information Retrieval, Libson, Portugal, 14/04/2020. https://doi.org/10.1007/978-3-030-45439-5_33

Interactive Learning for Multimedia at Large. / Khan, Omar Shahbaz ; Jónsson, Björn Thór; Rudinac, Stevan et al.
Advances in Information Retrieval: 42nd European Conference on IR Research, ECIR 2020. Lisbon, Portugal: Springer, 2020. p. 495-510 (Lecture Notes in Computer Science, Vol. 12035).

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

TY - GEN

T1 - Interactive Learning for Multimedia at Large

AU - Khan, Omar Shahbaz

AU - Jónsson, Björn Thór

AU - Rudinac, Stevan

AU - Zahálka, Jan

AU - Ragnarsdóttir, Hanna

AU - Þorleiksdóttir, Þórhildur

AU - Guðmundsson, Gylfi Þór

AU - Amsaleg, Laurent

AU - Worring, Marcel

N1 - Conference code: 42nd

PY - 2020/4

Y1 - 2020/4

N2 - Interactive learning has been suggested as a key method for addressing analytic multimedia tasks arising in several domains. Until recently, however, methods to maintain interactive performance at the scale of today’s media collections have not been addressed. We propose an interactive learning approach that builds on and extends the state of the art in user relevance feedback systems and high-dimensional indexing for multimedia. We report on a detailed experimental study using the ImageNet and YFCC100M collections, containing 14 million and 100 million images respectively. The proposed approach outperforms the relevant state-of-the-art approaches in terms of interactive performance, while improving suggestion relevance in some cases. In particular, even on YFCC100M, our approach requires less than 0.3 s per interaction round to generate suggestions, using a single computing core and less than 7 GB of main memory.

AB - Interactive learning has been suggested as a key method for addressing analytic multimedia tasks arising in several domains. Until recently, however, methods to maintain interactive performance at the scale of today’s media collections have not been addressed. We propose an interactive learning approach that builds on and extends the state of the art in user relevance feedback systems and high-dimensional indexing for multimedia. We report on a detailed experimental study using the ImageNet and YFCC100M collections, containing 14 million and 100 million images respectively. The proposed approach outperforms the relevant state-of-the-art approaches in terms of interactive performance, while improving suggestion relevance in some cases. In particular, even on YFCC100M, our approach requires less than 0.3 s per interaction round to generate suggestions, using a single computing core and less than 7 GB of main memory.

KW - Large multimedia collections

KW - Interactive multimodal learning

KW - YFCC100M

KW - High-dimensional indexing

KW - Large multimedia collections

KW - Interactive multimodal learning

KW - YFCC100M

KW - High-dimensional indexing

U2 - 10.1007/978-3-030-45439-5_33

DO - 10.1007/978-3-030-45439-5_33

M3 - Article in proceedings

SN - 978-3-030-45438-8

T3 - Lecture Notes in Computer Science

SP - 495

EP - 510

BT - Advances in Information Retrieval

PB - Springer

CY - Lisbon, Portugal

T2 - European Conference on Information Retrieval

Y2 - 14 April 2020 through 17 April 2020

ER -

Interactive Learning for Multimedia at Large

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this