We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: MarIA: Spanish Language Models

Abstract: This work presents MarIA, a family of Spanish language models and associated resources made available to the industry and the research community. Currently, MarIA includes RoBERTa-base, RoBERTa-large, GPT2 and GPT2-large Spanish language models, which can arguably be presented as the largest and most proficient language models in Spanish. The models were pretrained using a massive corpus of 570GB of clean and deduplicated texts with 135 billion words extracted from the Spanish Web Archive crawled by the National Library of Spain between 2009 and 2019. We assessed the performance of the models with nine existing evaluation datasets and with a novel extractive Question Answering dataset created ex novo. Overall, MarIA models outperform the existing Spanish models across a variety of NLU tasks and training settings.
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Journal reference: Procesamiento del Lenguaje Natural, v. 68, p. 39-60, mar. 2022. ISSN 1989-7553
DOI: 10.26342/2022-68-3
Cite as: arXiv:2107.07253 [cs.CL]
  (or arXiv:2107.07253v5 [cs.CL] for this version)

Submission history

From: Asier Gutiérrez-Fandiño [view email]
[v1] Thu, 15 Jul 2021 11:23:05 GMT (20kb)
[v2] Fri, 13 Aug 2021 13:47:44 GMT (23kb)
[v3] Fri, 1 Apr 2022 13:03:32 GMT (99kb,D)
[v4] Mon, 4 Apr 2022 16:25:12 GMT (99kb,D)
[v5] Tue, 5 Apr 2022 11:13:46 GMT (99kb,D)

Link back to: arXiv, form interface, contact.