Benchmarking Domain-specific Expert Search Using Workshop Programme Committees

Georgeta Bordea; Toine Bogers; Paul Buitelaar

doi:10.1145/2508497.2508501

Benchmarking Domain-specific Expert Search Using Workshop Programme Committees

Georgeta Bordea
, Toine Bogers
, Paul Buitelaar

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Abstract

Traditionally, relevance assessments for expert search have been gathered through self-assessment or based on the opinions of co-workers. We introduce three benchmark datasets for expert search that use conference workshops for relevance assessment. Our data sets cover entire research domains as opposed to single institutions. In addition, they provide a larger number of topic-person associations and allow a more objective and fine-grained evaluation of expertise than existing data sets do. We present and discuss baseline results for a language modelling and a topic-centric approach to expert search. We find that the topic-centric approach achieves the best results on domain-specific datasets.

Original language	English
Title of host publication	CompSci '13: Proceedings of the 2013 workshop on Computational scientometrics: Theory & Applications
Publisher	Association for Computing Machinery
Publication date	28 Oct 2013
Pages	19-24
ISBN (Print)	978-1-4503-2414-4
DOIs	https://doi.org/10.1145/2508497.2508501
Publication status	Published - 28 Oct 2013
Externally published	Yes
Event	ACM International Conference on Information and Knowledge Management - San Francisco, CA, United States Duration: 27 Oct 2013 → 1 Nov 2013 Conference number: 22 http://www.cikm2013.org/

Conference

Conference	ACM International Conference on Information and Knowledge Management
Number	22
Country/Territory	United States
City	San Francisco, CA
Period	27/10/2013 → 01/11/2013
Internet address	http://www.cikm2013.org/

Keywords

expert search
relevance assessment
benchmark datasets
research domains
evaluation of expertise

Access to Document

10.1145/2508497.2508501

compsci2013-benchmarkingFinal published version, 378 KB

http://dl.acm.org/citation.cfm?id=2508501

Cite this

@inproceedings{9de6036586814d38961043fa2341ed4d,

title = "Benchmarking Domain-specific Expert Search Using Workshop Programme Committees",

abstract = "Traditionally, relevance assessments for expert search have been gathered through self-assessment or based on the opinions of co-workers. We introduce three benchmark datasets for expert search that use conference workshops for relevance assessment. Our data sets cover entire research domains as opposed to single institutions. In addition, they provide a larger number of topic-person associations and allow a more objective and fine-grained evaluation of expertise than existing data sets do. We present and discuss baseline results for a language modelling and a topic-centric approach to expert search. We find that the topic-centric approach achieves the best results on domain-specific datasets.",

keywords = "expert search, relevance assessment, benchmark datasets, research domains, evaluation of expertise, expert search, relevance assessment, benchmark datasets, research domains, evaluation of expertise",

author = "Georgeta Bordea and Toine Bogers and Paul Buitelaar",

year = "2013",

month = oct,

day = "28",

doi = "10.1145/2508497.2508501",

language = "English",

isbn = "978-1-4503-2414-4",

pages = "19--24",

booktitle = "CompSci '13: Proceedings of the 2013 workshop on Computational scientometrics: Theory \& Applications",

publisher = "Association for Computing Machinery",

address = "United States",

note = "ACM International Conference on Information and Knowledge Management, CIKM 2013 ; Conference date: 27-10-2013 Through 01-11-2013",

url = "http://www.cikm2013.org/",

}

Bordea, G, Bogers, T & Buitelaar, P 2013, Benchmarking Domain-specific Expert Search Using Workshop Programme Committees. in CompSci '13: Proceedings of the 2013 workshop on Computational scientometrics: Theory & Applications. Association for Computing Machinery, pp. 19-24, ACM International Conference on Information and Knowledge Management, San Francisco, CA, United States, 27/10/2013. https://doi.org/10.1145/2508497.2508501

Benchmarking Domain-specific Expert Search Using Workshop Programme Committees. / Bordea, Georgeta; Bogers, Toine; Buitelaar, Paul.
CompSci '13: Proceedings of the 2013 workshop on Computational scientometrics: Theory & Applications. Association for Computing Machinery, 2013. p. 19-24.

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

TY - GEN

T1 - Benchmarking Domain-specific Expert Search Using Workshop Programme Committees

AU - Bordea, Georgeta

AU - Bogers, Toine

AU - Buitelaar, Paul

N1 - Conference code: 22

PY - 2013/10/28

Y1 - 2013/10/28

N2 - Traditionally, relevance assessments for expert search have been gathered through self-assessment or based on the opinions of co-workers. We introduce three benchmark datasets for expert search that use conference workshops for relevance assessment. Our data sets cover entire research domains as opposed to single institutions. In addition, they provide a larger number of topic-person associations and allow a more objective and fine-grained evaluation of expertise than existing data sets do. We present and discuss baseline results for a language modelling and a topic-centric approach to expert search. We find that the topic-centric approach achieves the best results on domain-specific datasets.

AB - Traditionally, relevance assessments for expert search have been gathered through self-assessment or based on the opinions of co-workers. We introduce three benchmark datasets for expert search that use conference workshops for relevance assessment. Our data sets cover entire research domains as opposed to single institutions. In addition, they provide a larger number of topic-person associations and allow a more objective and fine-grained evaluation of expertise than existing data sets do. We present and discuss baseline results for a language modelling and a topic-centric approach to expert search. We find that the topic-centric approach achieves the best results on domain-specific datasets.

KW - expert search

KW - relevance assessment

KW - benchmark datasets

KW - research domains

KW - evaluation of expertise

KW - expert search

KW - relevance assessment

KW - benchmark datasets

KW - research domains

KW - evaluation of expertise

U2 - 10.1145/2508497.2508501

DO - 10.1145/2508497.2508501

M3 - Article in proceedings

SN - 978-1-4503-2414-4

SP - 19

EP - 24

BT - CompSci '13: Proceedings of the 2013 workshop on Computational scientometrics: Theory & Applications

PB - Association for Computing Machinery

T2 - ACM International Conference on Information and Knowledge Management

Y2 - 27 October 2013 through 1 November 2013

ER -

Benchmarking Domain-specific Expert Search Using Workshop Programme Committees

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this