Impact of Interaction Strategies on User Relevance Feedback

Omar Shahbaz Khan, Björn Thór Jónsson, Jan Zahálka, Stevan Rudinac, Marcel Worring

Research output: Conference Article in Proceeding or Book/Report chapterArticle in proceedingsResearchpeer-review


User Relevance Feedback (URF) is a class of interactive learning methods that rely on the interaction between a human user and a system to analyze a media collection. To improve URF system evaluation and design better systems, it is important to understand the impact that different interaction strategies can have. Based on the literature and observations from real user sessions from the Lifelog Search Challenge and Video Browser Showdown, we analyze interaction strategies related to (a) labeling positive and negative examples, and (b) applying filters based on users' domain knowledge. Experiments show that there is no single optimal labeling strategy, as the best strategy depends on both the collection and the task. In particular, our results refute the common assumption that providing more training examples is always beneficial: strategies with a smaller number of prototypical examples lead to better results in some cases. We further observe that while expert filtering is unsurprisingly beneficial, aggressive filtering, especially by novice users, can hinder the completion of tasks. Finally, we observe that combining URF with filters leads to better results than using filters alone.
Original languageEnglish
Title of host publicationICMR '21: Proceedings of the 2021 International Conference on Multimedia Retrieval
Number of pages9
Place of PublicationTaipei, Taiwan (virtual)
PublisherAssociation for Computing Machinery
Publication dateAug 2021
ISBN (Print)978-1-4503-8463-6
Publication statusPublished - Aug 2021


Dive into the research topics of 'Impact of Interaction Strategies on User Relevance Feedback'. Together they form a unique fingerprint.

Cite this