The ROOTS Search Tool: Data Transparency for LLMs

Aleksandra Piktus; Christopher Akiki; Paulo Villegas; Hugo Laurençon; Gérard Dupont; Sasha Luccioni; Yacine Jernite; Anna Rogers

doi:10.18653/v1/2023.acl-demo.29

The ROOTS Search Tool: Data Transparency for LLMs

Aleksandra Piktus, Christopher Akiki, Paulo Villegas, Hugo Laurençon, Gérard Dupont, Sasha Luccioni, Yacine Jernite, Anna Rogers

Publikation: Konference artikel i Proceeding eller bog/rapport kapitel › Konferencebidrag i proceedings › Forskning › peer review

Abstract

ROOTS is a 1.6TB multilingual text corpus developed for the training of BLOOM, currently the largest language model explicitly accompanied by commensurate data governance efforts. In continuation of these efforts, we present the ROOTS Search Tool: a search engine over the entire ROOTS corpus offering both fuzzy and exact search capabilities. ROOTS is the largest corpus to date that can be investigated this way. The ROOTS Search Tool is open-sourced and available on Hugging Face Spaces: https://huggingface.co/spaces/bigscience-data/roots-search. We describe our implementation and the possible use cases of our tool.

Originalsprog	Engelsk
Titel	Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)
Antal sider	11
Udgivelsessted	Toronto, Canada
Forlag	Association for Computational Linguistics
Publikationsdato	1 jul. 2023
Sider	304-314
DOI	https://doi.org/10.18653/v1/2023.acl-demo.29
Status	Udgivet - 1 jul. 2023

Emneord

Multilingual Text Corpus
Language Model Training
Data Governance
Search Engine
Open-Source Tool

Adgang til dokumentet

10.18653/v1/2023.acl-demo.29

Citationsformater

Piktus, A., Akiki, C., Villegas, P., Laurençon, H., Dupont, G., Luccioni, S., Jernite, Y., & Rogers, A. (2023). The ROOTS Search Tool: Data Transparency for LLMs. I Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations) (s. 304-314). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.acl-demo.29

@inproceedings{cac34f53395843c5808fd4ff52db8f8b,

title = "The ROOTS Search Tool: Data Transparency for LLMs",

abstract = "ROOTS is a 1.6TB multilingual text corpus developed for the training of BLOOM, currently the largest language model explicitly accompanied by commensurate data governance efforts. In continuation of these efforts, we present the ROOTS Search Tool: a search engine over the entire ROOTS corpus offering both fuzzy and exact search capabilities. ROOTS is the largest corpus to date that can be investigated this way. The ROOTS Search Tool is open-sourced and available on Hugging Face Spaces: https://huggingface.co/spaces/bigscience-data/roots-search. We describe our implementation and the possible use cases of our tool.",

keywords = "Multilingual Text Corpus, Language Model Training, Data Governance, Search Engine, Open-Source Tool, Multilingual Text Corpus, Language Model Training, Data Governance, Search Engine, Open-Source Tool",

author = "Aleksandra Piktus and Christopher Akiki and Paulo Villegas and Hugo Lauren{\c c}on and G{\'e}rard Dupont and Sasha Luccioni and Yacine Jernite and Anna Rogers",

year = "2023",

month = jul,

day = "1",

doi = "10.18653/v1/2023.acl-demo.29",

language = "English",

pages = "304--314",

booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)",

publisher = "Association for Computational Linguistics",

address = "United States",

}

Piktus, A, Akiki, C, Villegas, P, Laurençon, H, Dupont, G, Luccioni, S, Jernite, Y & Rogers, A 2023, The ROOTS Search Tool: Data Transparency for LLMs. i Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations). Association for Computational Linguistics, Toronto, Canada, s. 304-314. https://doi.org/10.18653/v1/2023.acl-demo.29

The ROOTS Search Tool: Data Transparency for LLMs. / Piktus, Aleksandra; Akiki, Christopher; Villegas, Paulo et al.
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations). Toronto, Canada: Association for Computational Linguistics, 2023. s. 304-314.

Publikation: Konference artikel i Proceeding eller bog/rapport kapitel › Konferencebidrag i proceedings › Forskning › peer review

TY - GEN

T1 - The ROOTS Search Tool: Data Transparency for LLMs

AU - Piktus, Aleksandra

AU - Akiki, Christopher

AU - Villegas, Paulo

AU - Laurençon, Hugo

AU - Dupont, Gérard

AU - Luccioni, Sasha

AU - Jernite, Yacine

AU - Rogers, Anna

PY - 2023/7/1

Y1 - 2023/7/1

N2 - ROOTS is a 1.6TB multilingual text corpus developed for the training of BLOOM, currently the largest language model explicitly accompanied by commensurate data governance efforts. In continuation of these efforts, we present the ROOTS Search Tool: a search engine over the entire ROOTS corpus offering both fuzzy and exact search capabilities. ROOTS is the largest corpus to date that can be investigated this way. The ROOTS Search Tool is open-sourced and available on Hugging Face Spaces: https://huggingface.co/spaces/bigscience-data/roots-search. We describe our implementation and the possible use cases of our tool.

AB - ROOTS is a 1.6TB multilingual text corpus developed for the training of BLOOM, currently the largest language model explicitly accompanied by commensurate data governance efforts. In continuation of these efforts, we present the ROOTS Search Tool: a search engine over the entire ROOTS corpus offering both fuzzy and exact search capabilities. ROOTS is the largest corpus to date that can be investigated this way. The ROOTS Search Tool is open-sourced and available on Hugging Face Spaces: https://huggingface.co/spaces/bigscience-data/roots-search. We describe our implementation and the possible use cases of our tool.

KW - Multilingual Text Corpus

KW - Language Model Training

KW - Data Governance

KW - Search Engine

KW - Open-Source Tool

KW - Multilingual Text Corpus

KW - Language Model Training

KW - Data Governance

KW - Search Engine

KW - Open-Source Tool

U2 - 10.18653/v1/2023.acl-demo.29

DO - 10.18653/v1/2023.acl-demo.29

M3 - Article in proceedings

SP - 304

EP - 314

BT - Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)

PB - Association for Computational Linguistics

CY - Toronto, Canada

ER -

The ROOTS Search Tool: Data Transparency for LLMs

Abstract

Emneord

Adgang til dokumentet

Fingeraftryk

Citationsformater