References & Citations
Computer Science > Computation and Language
Title: Multilingual Culture-Independent Word Analogy Datasets
(Submitted on 22 Nov 2019 (v1), last revised 27 Mar 2020 (this version, v2))
Abstract: In text processing, deep neural networks mostly use word embeddings as an input. Embeddings have to ensure that relations between words are reflected through distances in a high-dimensional numeric space. To compare the quality of different text embeddings, typically, we use benchmark datasets. We present a collection of such datasets for the word analogy task in nine languages: Croatian, English, Estonian, Finnish, Latvian, Lithuanian, Russian, Slovenian, and Swedish. We redesigned the original monolingual analogy task to be much more culturally independent and also constructed cross-lingual analogy datasets for the involved languages. We present basic statistics of the created datasets and their initial evaluation using fastText embeddings.
Submission history
From: Matej Ulčar [view email][v1] Fri, 22 Nov 2019 13:39:06 GMT (21kb)
[v2] Fri, 27 Mar 2020 15:32:16 GMT (26kb)
Link back to: arXiv, form interface, contact.