We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Scalable Cross-lingual Document Similarity through Language-specific Concept Hierarchies

Abstract: With the ongoing growth in number of digital articles in a wider set of languages and the expanding use of different languages, we need annotation methods that enable browsing multi-lingual corpora. Multilingual probabilistic topic models have recently emerged as a group of semi-supervised machine learning models that can be used to perform thematic explorations on collections of texts in multiple languages. However, these approaches require theme-aligned training data to create a language-independent space. This constraint limits the amount of scenarios that this technique can offer solutions to train and makes it difficult to scale up to situations where a huge collection of multi-lingual documents are required during the training phase. This paper presents an unsupervised document similarity algorithm that does not require parallel or comparable corpora, or any other type of translation resource. The algorithm annotates topics automatically created from documents in a single language with cross-lingual labels and describes documents by hierarchies of multi-lingual concepts from independently-trained models. Experiments performed on the English, Spanish and French editions of JCR-Acquis corpora reveal promising results on classifying and sorting documents by similar content.
Comments: Accepted at the 10th International Conference on Knowledge Capture (K-CAP 2019)
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
Journal reference: AACM Proceedings of the 10th International Conference on Knowledge Capture, pages = 147-153, K-CAP 19 (2020)
DOI: 10.1145/3360901.3364444
Cite as: arXiv:2101.03026 [cs.CL]
  (or arXiv:2101.03026v1 [cs.CL] for this version)

Submission history

From: Carlos Badenes-Olmedo [view email]
[v1] Tue, 15 Dec 2020 10:42:40 GMT (1850kb,D)

Link back to: arXiv, form interface, contact.