Benchmarking Domain-specific Expert Search Using Workshop Programme Committees

Georgeta Bordea; Toine Bogers; Paul Buitelaar

doi:10.1145/2508497.2508501

Benchmarking Domain-specific Expert Search Using Workshop Programme Committees

Georgeta Bordea, Toine Bogers, Paul Buitelaar

Publikation: Konference artikel i Proceeding eller bog/rapport kapitel › Konferencebidrag i proceedings › Forskning › peer review

Abstract

Traditionally, relevance assessments for expert search have been gathered through self-assessment or based on the opinions of co-workers. We introduce three benchmark datasets for expert search that use conference workshops for relevance assessment. Our data sets cover entire research domains as opposed to single institutions. In addition, they provide a larger number of topic-person associations and allow a more objective and fine-grained evaluation of expertise than existing data sets do. We present and discuss baseline results for a language modelling and a topic-centric approach to expert search. We find that the topic-centric approach achieves the best results on domain-specific datasets.

Originalsprog	Engelsk
Titel	CompSci '13: Proceedings of the 2013 workshop on Computational scientometrics: Theory & Applications
Forlag	Association for Computing Machinery
Publikationsdato	28 okt. 2013
Sider	19-24
ISBN (Trykt)	978-1-4503-2414-4
DOI	https://doi.org/10.1145/2508497.2508501
Status	Udgivet - 28 okt. 2013
Udgivet eksternt	Ja
Begivenhed	22nd ACM International Conference on Information and Knowledge Management - San Francisco, CA, USA Varighed: 27 okt. 2013 → 1 nov. 2013 Konferencens nummer: 22 http://www.cikm2013.org/

Konference

Konference	22nd ACM International Conference on Information and Knowledge Management
Nummer	22
Land/Område	USA
By	San Francisco, CA
Periode	27/10/2013 → 01/11/2013
Internetadresse	http://www.cikm2013.org/

Emneord

expert search
relevance assessment
benchmark datasets
research domains
evaluation of expertise

Adgang til dokumentet

10.1145/2508497.2508501

compsci2013-benchmarkingForlagets udgivne version, 378 KB

http://dl.acm.org/citation.cfm?id=2508501

Citationsformater

@inproceedings{9de6036586814d38961043fa2341ed4d,

title = "Benchmarking Domain-specific Expert Search Using Workshop Programme Committees",

abstract = "Traditionally, relevance assessments for expert search have been gathered through self-assessment or based on the opinions of co-workers. We introduce three benchmark datasets for expert search that use conference workshops for relevance assessment. Our data sets cover entire research domains as opposed to single institutions. In addition, they provide a larger number of topic-person associations and allow a more objective and fine-grained evaluation of expertise than existing data sets do. We present and discuss baseline results for a language modelling and a topic-centric approach to expert search. We find that the topic-centric approach achieves the best results on domain-specific datasets.",

keywords = "expert search, relevance assessment, benchmark datasets, research domains, evaluation of expertise, expert search, relevance assessment, benchmark datasets, research domains, evaluation of expertise",

author = "Georgeta Bordea and Toine Bogers and Paul Buitelaar",

year = "2013",

month = oct,

day = "28",

doi = "10.1145/2508497.2508501",

language = "English",

isbn = "978-1-4503-2414-4",

pages = "19--24",

booktitle = "CompSci '13: Proceedings of the 2013 workshop on Computational scientometrics: Theory \& Applications",

publisher = "Association for Computing Machinery",

address = "United States",

note = "ACM International Conference on Information and Knowledge Management, CIKM 2013 ; Conference date: 27-10-2013 Through 01-11-2013",

url = "http://www.cikm2013.org/",

}

Bordea, G, Bogers, T & Buitelaar, P 2013, Benchmarking Domain-specific Expert Search Using Workshop Programme Committees. i CompSci '13: Proceedings of the 2013 workshop on Computational scientometrics: Theory & Applications. Association for Computing Machinery, s. 19-24, 22nd ACM International Conference on Information and Knowledge Management, San Francisco, CA, USA, 27/10/2013. https://doi.org/10.1145/2508497.2508501

Benchmarking Domain-specific Expert Search Using Workshop Programme Committees. / Bordea, Georgeta; Bogers, Toine; Buitelaar, Paul.
CompSci '13: Proceedings of the 2013 workshop on Computational scientometrics: Theory & Applications. Association for Computing Machinery, 2013. s. 19-24.

Publikation: Konference artikel i Proceeding eller bog/rapport kapitel › Konferencebidrag i proceedings › Forskning › peer review

TY - GEN

T1 - Benchmarking Domain-specific Expert Search Using Workshop Programme Committees

AU - Bordea, Georgeta

AU - Bogers, Toine

AU - Buitelaar, Paul

N1 - Conference code: 22

PY - 2013/10/28

Y1 - 2013/10/28

N2 - Traditionally, relevance assessments for expert search have been gathered through self-assessment or based on the opinions of co-workers. We introduce three benchmark datasets for expert search that use conference workshops for relevance assessment. Our data sets cover entire research domains as opposed to single institutions. In addition, they provide a larger number of topic-person associations and allow a more objective and fine-grained evaluation of expertise than existing data sets do. We present and discuss baseline results for a language modelling and a topic-centric approach to expert search. We find that the topic-centric approach achieves the best results on domain-specific datasets.

AB - Traditionally, relevance assessments for expert search have been gathered through self-assessment or based on the opinions of co-workers. We introduce three benchmark datasets for expert search that use conference workshops for relevance assessment. Our data sets cover entire research domains as opposed to single institutions. In addition, they provide a larger number of topic-person associations and allow a more objective and fine-grained evaluation of expertise than existing data sets do. We present and discuss baseline results for a language modelling and a topic-centric approach to expert search. We find that the topic-centric approach achieves the best results on domain-specific datasets.

KW - expert search

KW - relevance assessment

KW - benchmark datasets

KW - research domains

KW - evaluation of expertise

KW - expert search

KW - relevance assessment

KW - benchmark datasets

KW - research domains

KW - evaluation of expertise

U2 - 10.1145/2508497.2508501

DO - 10.1145/2508497.2508501

M3 - Article in proceedings

SN - 978-1-4503-2414-4

SP - 19

EP - 24

BT - CompSci '13: Proceedings of the 2013 workshop on Computational scientometrics: Theory & Applications

PB - Association for Computing Machinery

T2 - ACM International Conference on Information and Knowledge Management

Y2 - 27 October 2013 through 1 November 2013

ER -

Benchmarking Domain-specific Expert Search Using Workshop Programme Committees

Abstract

Konference

Emneord

Adgang til dokumentet

Fingeraftryk

Citationsformater