Benchmarking Domain-specific Expert Search Using Workshop Programme Committees

Georgeta Bordea, Toine Bogers, Paul Buitelaar

Research output: Conference Article in Proceeding or Book/Report chapterArticle in proceedingsResearchpeer-review


Traditionally, relevance assessments for expert search have been gathered through self-assessment or based on the opinions of co-workers. We introduce three benchmark datasets for expert search that use conference workshops for relevance assessment. Our data sets cover entire research domains as opposed to single institutions. In addition, they provide a larger number of topic-person associations and allow a more objective and fine-grained evaluation of expertise than existing data sets do. We present and discuss baseline results for a language modelling and a topic-centric approach to expert search. We find that the topic-centric approach achieves the best results on domain-specific datasets.
Original languageEnglish
Title of host publicationCompSci '13: Proceedings of the 2013 workshop on Computational scientometrics: Theory & Applications
PublisherAssociation for Computing Machinery
Publication date28 Oct 2013
ISBN (Print)978-1-4503-2414-4
Publication statusPublished - 28 Oct 2013
Externally publishedYes
EventACM International Conference on Information and Knowledge Management - San Francisco, CA, United States
Duration: 27 Oct 20131 Nov 2013
Conference number: 22


ConferenceACM International Conference on Information and Knowledge Management
Country/TerritoryUnited States
CitySan Francisco, CA
Internet address


  • expert search
  • relevance assessment
  • benchmark datasets
  • research domains
  • evaluation of expertise


Dive into the research topics of 'Benchmarking Domain-specific Expert Search Using Workshop Programme Committees'. Together they form a unique fingerprint.

Cite this