Running Experiments with Confidence and Sanity

Martin Aumüller; Matteo Ceccarello

doi:10.1007/978-3-030-60936-8_31

Running Experiments with Confidence and Sanity

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Abstract

Analyzing data from large experimental suites is a daily task for anyone doing experimental algorithmics. In this paper we report on several approaches we tried for this seemingly mundane task in a similarity search setting, reflecting on the challenges it poses.

We conclude by proposing a workflow, which can be implemented using several tools, that allows to analyze experimental data with confidence.

The extended version of this paper and the support code are provided at https://github.com/Cecca/running-experiments.

Original language	English
Title of host publication	International Conference on Similarity Search and Applications
Publisher	Springer
Publication date	2020
Pages	18-32
DOIs	https://doi.org/10.1007/978-3-030-60936-8_31
Publication status	Published - 2020

Keywords

Experimental algorithmics
Data analysis
Similarity search
Workflow development
Support code

Access to Document

10.1007/978-3-030-60936-8_31

running-experimentsAccepted author manuscript, 592 KB

Cite this

@inproceedings{df94bc0cec7f4976b35ef89c5dc3a69a,

title = "Running Experiments with Confidence and Sanity",

abstract = "Analyzing data from large experimental suites is a daily task for anyone doing experimental algorithmics. In this paper we report on several approaches we tried for this seemingly mundane task in a similarity search setting, reflecting on the challenges it poses.We conclude by proposing a workflow, which can be implemented using several tools, that allows to analyze experimental data with confidence.The extended version of this paper and the support code are provided at https://github.com/Cecca/running-experiments.",

keywords = "Experimental algorithmics, Data analysis, Similarity search, Workflow development, Support code, Experimental algorithmics, Data analysis, Similarity search, Workflow development, Support code",

author = "Martin Aum{\"u}ller and Matteo Ceccarello",

year = "2020",

doi = "10.1007/978-3-030-60936-8_31",

language = "English",

pages = "18--32",

booktitle = "International Conference on Similarity Search and Applications",

publisher = "Springer",

address = "Germany",

}

TY - GEN

T1 - Running Experiments with Confidence and Sanity

AU - Aumüller, Martin

AU - Ceccarello, Matteo

PY - 2020

Y1 - 2020

N2 - Analyzing data from large experimental suites is a daily task for anyone doing experimental algorithmics. In this paper we report on several approaches we tried for this seemingly mundane task in a similarity search setting, reflecting on the challenges it poses.We conclude by proposing a workflow, which can be implemented using several tools, that allows to analyze experimental data with confidence.The extended version of this paper and the support code are provided at https://github.com/Cecca/running-experiments.

AB - Analyzing data from large experimental suites is a daily task for anyone doing experimental algorithmics. In this paper we report on several approaches we tried for this seemingly mundane task in a similarity search setting, reflecting on the challenges it poses.We conclude by proposing a workflow, which can be implemented using several tools, that allows to analyze experimental data with confidence.The extended version of this paper and the support code are provided at https://github.com/Cecca/running-experiments.

KW - Experimental algorithmics

KW - Data analysis

KW - Similarity search

KW - Workflow development

KW - Support code

KW - Experimental algorithmics

KW - Data analysis

KW - Similarity search

KW - Workflow development

KW - Support code

U2 - 10.1007/978-3-030-60936-8_31

DO - 10.1007/978-3-030-60936-8_31

M3 - Article in proceedings

SP - 18

EP - 32

BT - International Conference on Similarity Search and Applications

PB - Springer

ER -

Running Experiments with Confidence and Sanity

Abstract

Keywords

Access to Document

Fingerprint

Cite this