Is the Reign of Interactive Search Eternal? Findings from the Video Browser Showdown 2020

Jakub Lokoč, Patrik Veselý, František Mejzlík, Gregor Kovalčík, Tomáš Souček, Luca Rossetto, Klaus Schoeffmann, Werner Bailer, Cathal Gurrin, Loris Sauter, Jaeyub Song, Stefanos Vrochidis, Jiaxin Wu, Björn Thór Jónsson

Research output: Journal Article or Conference Article in JournalJournal articleResearchpeer-review

Abstract

Comprehensive and fair performance evaluation of information retrieval systems represents an essential task for the current information age. Whereas Cranfield-based evaluations with benchmark datasets support development of retrieval models, significant evaluation efforts are required also for user-oriented systems that try to boost performance with an interactive search approach. This paper presents findings from the 9th Video Browser Showdown, a competition that focuses on a legitimate comparison of interactive search systems designed for challenging known-item search tasks over a large video collection. During previous installments of the competition, the interactive nature of participating systems was a key feature to satisfy known-item search needs and this paper continues to support this hypothesis. Despite the fact that top-performing systems integrate the most recent deep learning models into their retrieval process, interactive searching remains a necessary component of successful strategies for known-item search tasks. Alongside the description of competition settings, evaluated tasks, participating teams, and overall results, this paper presents a detailed analysis of query logs collected by the top three performing systems SOMHunter, VIRET, and vitrivr. The analysis provides a quantitative insight to the observed performance of the systems and constitutes a new baseline methodology for future events. The results reveal that the top two systems mostly relied on temporal queries before a correct frame was identified. An interaction log analysis complements the result log findings and points to the importance of result set and video browsing approaches. Finally, various outlooks are discussed in order to improve the Video Browser Showdown challenge in the future.
Original languageEnglish
Article number91
JournalACM Transactions on Multimedia Computing, Communications, and Applications
Volume17
Issue number3
Pages (from-to)91:1-91:26
Number of pages26
ISSN1551-6857
DOIs
Publication statusPublished - Aug 2021

Keywords

  • interactive video retrieval
  • deep learning
  • interactive search evaluation

Fingerprint

Dive into the research topics of 'Is the Reign of Interactive Search Eternal? Findings from the Video Browser Showdown 2020'. Together they form a unique fingerprint.

Cite this