Overview of the SISAP 2025 Indexing Challenge.

Eric Sadit Tellez; Edgar Chávez; Martin Aumüller; Vladimir Mic

Overview of the SISAP 2025 Indexing Challenge.

Eric Sadit Tellez
, Edgar Chávez
, Martin Aumüller
, Vladimir Mic

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Abstract

This paper summarizes the innovative solutions presented at the third edition of the SISAP Indexing Challenge held at SISAP 2025.
The challenge featured two distinct tasks involving vector embeddings derived from a large corpus using neural encoders. It proposed the following two tasks under strict memory and computational constraints:
– Task 1: Approximate nearest neighbor search achieving an average
recall of at least 0.7 for 30-NN, using out-of-distribution objects as
queries.
– Task 2: k-NN (k = 15) graph construction for large datasets, requiring an average recall of at least 0.8.
Both tasks required solutions to operate within strict resource limits: 16 GB of RAM, 8 virtual CPUs, and a 12-hour wall-clock time for the end-to-end pipeline (including data loading, pre-processing, indexing, and searching). Each task imposes different minimum quality requirements and ranking specifications. Participants developed strategies such as data compression, optimized indexing, and efficient search algorithms to meet these constraints. This paper details the challenge design, explains the evaluation framework, and provides an overview of the submitted solutions.

Original language	English
Title of host publication	SISAP
Number of pages	12
Publisher	Springer
Publication date	2025
Pages	403-414
ISBN (Print)	978-3-032-06068-6
ISBN (Electronic)	978-3-032-06069-3
Publication status	Published - 2025
Event	International Conference on Similarity Search and Applications - Reykjavik, Iceland Duration: 1 Oct 2025 → 3 Oct 2025 Conference number: 18 https://www.sisap.org/2025

Conference

Conference	International Conference on Similarity Search and Applications
Number	18
Country/Territory	Iceland
City	Reykjavik
Period	01/10/2025 → 03/10/2025
Internet address	https://www.sisap.org/2025

Keywords

Approximate nearest neighbor search
k-NN graph construction
Vector embeddings
Neural encoders
Evaluation framework

Access to Document

2025-sisap-overviewAccepted author manuscript, 422 KB

Cite this

@inproceedings{c0b3409a119b4b34a8fe12feecf9a8e8,

title = "Overview of the SISAP 2025 Indexing Challenge.",

abstract = "This paper summarizes the innovative solutions presented at the third edition of the SISAP Indexing Challenge held at SISAP 2025. The challenge featured two distinct tasks involving vector embeddings derived from a large corpus using neural encoders. It proposed the following two tasks under strict memory and computational constraints: – Task 1: Approximate nearest neighbor search achieving an average recall of at least 0.7 for 30-NN, using out-of-distribution objects as queries. – Task 2: k-NN (k = 15) graph construction for large datasets, requiring an average recall of at least 0.8. Both tasks required solutions to operate within strict resource limits: 16 GB of RAM, 8 virtual CPUs, and a 12-hour wall-clock time for the end-to-end pipeline (including data loading, pre-processing, indexing, and searching). Each task imposes different minimum quality requirements and ranking specifications. Participants developed strategies such as data compression, optimized indexing, and efficient search algorithms to meet these constraints. This paper details the challenge design, explains the evaluation framework, and provides an overview of the submitted solutions.",

keywords = "Approximate nearest neighbor search, k-NN graph construction, Vector embeddings, Neural encoders, Evaluation framework",

author = "Tellez, \{Eric Sadit\} and Edgar Ch{\'a}vez and Martin Aum{\"u}ller and Vladimir Mic",

note = "DBLP License: DBLP's bibliographic metadata records provided through http://dblp.org/ are distributed under a Creative Commons CC0 1.0 Universal Public Domain Dedication. Although the bibliographic metadata records are provided consistent with CC0 1.0 Dedication, the content described by the metadata records is not. Content may be subject to copyright, rights of privacy, rights of publicity and other restrictions.; International Conference on Similarity Search and Applications, SISAP ; Conference date: 01-10-2025 Through 03-10-2025",

year = "2025",

language = "English",

isbn = "978-3-032-06068-6",

pages = "403--414",

booktitle = "SISAP",

publisher = "Springer",

address = "Germany",

url = "https://www.sisap.org/2025",

}

TY - GEN

T1 - Overview of the SISAP 2025 Indexing Challenge.

AU - Tellez, Eric Sadit

AU - Chávez, Edgar

AU - Aumüller, Martin

AU - Mic, Vladimir

N1 - Conference code: 18

PY - 2025

Y1 - 2025

N2 - This paper summarizes the innovative solutions presented at the third edition of the SISAP Indexing Challenge held at SISAP 2025. The challenge featured two distinct tasks involving vector embeddings derived from a large corpus using neural encoders. It proposed the following two tasks under strict memory and computational constraints: – Task 1: Approximate nearest neighbor search achieving an average recall of at least 0.7 for 30-NN, using out-of-distribution objects as queries. – Task 2: k-NN (k = 15) graph construction for large datasets, requiring an average recall of at least 0.8. Both tasks required solutions to operate within strict resource limits: 16 GB of RAM, 8 virtual CPUs, and a 12-hour wall-clock time for the end-to-end pipeline (including data loading, pre-processing, indexing, and searching). Each task imposes different minimum quality requirements and ranking specifications. Participants developed strategies such as data compression, optimized indexing, and efficient search algorithms to meet these constraints. This paper details the challenge design, explains the evaluation framework, and provides an overview of the submitted solutions.

AB - This paper summarizes the innovative solutions presented at the third edition of the SISAP Indexing Challenge held at SISAP 2025. The challenge featured two distinct tasks involving vector embeddings derived from a large corpus using neural encoders. It proposed the following two tasks under strict memory and computational constraints: – Task 1: Approximate nearest neighbor search achieving an average recall of at least 0.7 for 30-NN, using out-of-distribution objects as queries. – Task 2: k-NN (k = 15) graph construction for large datasets, requiring an average recall of at least 0.8. Both tasks required solutions to operate within strict resource limits: 16 GB of RAM, 8 virtual CPUs, and a 12-hour wall-clock time for the end-to-end pipeline (including data loading, pre-processing, indexing, and searching). Each task imposes different minimum quality requirements and ranking specifications. Participants developed strategies such as data compression, optimized indexing, and efficient search algorithms to meet these constraints. This paper details the challenge design, explains the evaluation framework, and provides an overview of the submitted solutions.

KW - Approximate nearest neighbor search

KW - k-NN graph construction

KW - Vector embeddings

KW - Neural encoders

KW - Evaluation framework

M3 - Article in proceedings

SN - 978-3-032-06068-6

SP - 403

EP - 414

BT - SISAP

PB - Springer

T2 - International Conference on Similarity Search and Applications

Y2 - 1 October 2025 through 3 October 2025

ER -

Overview of the SISAP 2025 Indexing Challenge.

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this