Exquisitor at the Video Browser Showdown 2022

Omar Shahbaz Khan; Ujjwal Sharma; Björn Þór Jónsson; Dennis C. Koelma; Stevan Rudinac; Marcel Worring; Jan Zahálka

doi:10.1007/978-3-030-98355-0_47

Exquisitor at the Video Browser Showdown 2022

Omar Shahbaz Khan
, Ujjwal Sharma
, Björn Þór Jónsson
, Dennis C. Koelma
, Stevan Rudinac
, Marcel Worring
, Jan Zahálka

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Abstract

Exquisitor is the state-of-the-art large-scale interactive learning approach for media exploration that utilizes user relevance feedback at its core and is capable of interacting with collections containing more than 100M multimedia items at sub-second latency. In this work, we propose improvements to Exquisitor that include new features extracted at shot level for semantic concepts, scenes and actions. In addition, we introduce extensions to the video summary interface providing a better overview of the shots. Finally, we replace a simple keyword search featured in the previous versions of the system with a semantic search based on modern contextual representations.

Original language	English
Title of host publication	MultiMedia Modeling
Editors	Björn Þór Jónsson, Cathal Gurrin, Minh-Triet Tran, Duc-Tien Dang-Nguyen, Anita Min-Chun Hu, Binh Huynh Thi Thanh, Benoit Huet
Number of pages	7
Place of Publication	Cham
Publisher	Springer
Publication date	15 Mar 2022
Pages	511-517
ISBN (Print)	978-3-030-98355-0
DOIs	https://doi.org/10.1007/978-3-030-98355-0_47
Publication status	Published - 15 Mar 2022
Event	Multimedia modeling - Cua Duong Village, Phu Quoc, Viet Nam Duration: 6 Jun 2022 → 10 Jun 2022 Conference number: 28th https://mmm2022.org/

Conference

Conference	Multimedia modeling
Number	28th
Location	Cua Duong Village
Country/Territory	Viet Nam
City	Phu Quoc
Period	06/06/2022 → 10/06/2022
Internet address	https://mmm2022.org/

Series	Lecture Notes in Computer Science
Volume	13142
ISSN	0302-9743

Keywords

Interactive learning
Video browsing
Multimodal representation learning
Semantic search

Access to Document

10.1007/978-3-030-98355-0_47

Cite this

Khan, Omar Shahbaz ; Sharma, Ujjwal ; Jónsson, Björn Þór et al. / Exquisitor at the Video Browser Showdown 2022. MultiMedia Modeling. editor / Björn Þór Jónsson ; Cathal Gurrin ; Minh-Triet Tran ; Duc-Tien Dang-Nguyen ; Anita Min-Chun Hu ; Binh Huynh Thi Thanh ; Benoit Huet. Cham : Springer, 2022. pp. 511-517 (Lecture Notes in Computer Science, Vol. 13142).

@inproceedings{ae39e90982c94c4e9686c86547f948ab,

title = "Exquisitor at the Video Browser Showdown 2022",

abstract = "Exquisitor is the state-of-the-art large-scale interactive learning approach for media exploration that utilizes user relevance feedback at its core and is capable of interacting with collections containing more than 100M multimedia items at sub-second latency. In this work, we propose improvements to Exquisitor that include new features extracted at shot level for semantic concepts, scenes and actions. In addition, we introduce extensions to the video summary interface providing a better overview of the shots. Finally, we replace a simple keyword search featured in the previous versions of the system with a semantic search based on modern contextual representations.",

keywords = "Interactive learning, Video browsing, Multimodal representation learning, Semantic search",

author = "Khan, \{Omar Shahbaz\} and Ujjwal Sharma and J{\'o}nsson, \{Bj{\"o}rn {\TH}{\'o}r\} and Koelma, \{Dennis C.\} and Stevan Rudinac and Marcel Worring and Jan Zah{\'a}lka",

year = "2022",

month = mar,

day = "15",

doi = "10.1007/978-3-030-98355-0\_47",

language = "English",

isbn = "978-3-030-98355-0",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "511--517",

editor = "\{{\TH}{\'o}r J{\'o}nsson\}, Bj{\"o}rn and Cathal Gurrin and Minh-Triet Tran and Duc-Tien Dang-Nguyen and Hu, \{Anita Min-Chun\} and \{Huynh Thi Thanh\}, Binh and Benoit Huet",

booktitle = "MultiMedia Modeling",

address = "Germany",

note = "Multimedia modeling ; Conference date: 06-06-2022 Through 10-06-2022",

url = "https://mmm2022.org/",

}

Khan, OS, Sharma, U, Jónsson, BÞ, Koelma, DC, Rudinac, S, Worring, M & Zahálka, J 2022, Exquisitor at the Video Browser Showdown 2022. in B Þór Jónsson, C Gurrin, M-T Tran, D-T Dang-Nguyen, AM-C Hu, B Huynh Thi Thanh & B Huet (eds), MultiMedia Modeling. Springer, Cham, Lecture Notes in Computer Science, vol. 13142, pp. 511-517, Multimedia modeling , Phu Quoc, Viet Nam, 06/06/2022. https://doi.org/10.1007/978-3-030-98355-0_47

Exquisitor at the Video Browser Showdown 2022. / Khan, Omar Shahbaz; Sharma, Ujjwal; Jónsson, Björn Þór et al.
MultiMedia Modeling. ed. / Björn Þór Jónsson; Cathal Gurrin; Minh-Triet Tran; Duc-Tien Dang-Nguyen; Anita Min-Chun Hu; Binh Huynh Thi Thanh; Benoit Huet. Cham: Springer, 2022. p. 511-517 (Lecture Notes in Computer Science, Vol. 13142).

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

TY - GEN

T1 - Exquisitor at the Video Browser Showdown 2022

AU - Khan, Omar Shahbaz

AU - Sharma, Ujjwal

AU - Jónsson, Björn Þór

AU - Koelma, Dennis C.

AU - Rudinac, Stevan

AU - Worring, Marcel

AU - Zahálka, Jan

N1 - Conference code: 28th

PY - 2022/3/15

Y1 - 2022/3/15

N2 - Exquisitor is the state-of-the-art large-scale interactive learning approach for media exploration that utilizes user relevance feedback at its core and is capable of interacting with collections containing more than 100M multimedia items at sub-second latency. In this work, we propose improvements to Exquisitor that include new features extracted at shot level for semantic concepts, scenes and actions. In addition, we introduce extensions to the video summary interface providing a better overview of the shots. Finally, we replace a simple keyword search featured in the previous versions of the system with a semantic search based on modern contextual representations.

AB - Exquisitor is the state-of-the-art large-scale interactive learning approach for media exploration that utilizes user relevance feedback at its core and is capable of interacting with collections containing more than 100M multimedia items at sub-second latency. In this work, we propose improvements to Exquisitor that include new features extracted at shot level for semantic concepts, scenes and actions. In addition, we introduce extensions to the video summary interface providing a better overview of the shots. Finally, we replace a simple keyword search featured in the previous versions of the system with a semantic search based on modern contextual representations.

KW - Interactive learning

KW - Video browsing

KW - Multimodal representation learning

KW - Semantic search

U2 - 10.1007/978-3-030-98355-0_47

DO - 10.1007/978-3-030-98355-0_47

M3 - Article in proceedings

SN - 978-3-030-98355-0

T3 - Lecture Notes in Computer Science

SP - 511

EP - 517

BT - MultiMedia Modeling

A2 - Þór Jónsson, Björn

A2 - Gurrin, Cathal

A2 - Tran, Minh-Triet

A2 - Dang-Nguyen, Duc-Tien

A2 - Hu, Anita Min-Chun

A2 - Huynh Thi Thanh, Binh

A2 - Huet, Benoit

PB - Springer

CY - Cham

T2 - Multimedia modeling

Y2 - 6 June 2022 through 10 June 2022