References & Citations
Computer Science > Computation and Language
Title: BERTifying Sinhala -- A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classification
(Submitted on 16 Aug 2022 (v1), last revised 17 Aug 2022 (this version, v2))
Abstract: This research provides the first comprehensive analysis of the performance of pre-trained language models for Sinhala text classification. We test on a set of different Sinhala text classification tasks and our analysis shows that out of the pre-trained multilingual models that include Sinhala (XLM-R, LaBSE, and LASER), XLM-R is the best model by far for Sinhala text classification. We also pre-train two RoBERTa-based monolingual Sinhala models, which are far superior to the existing pre-trained language models for Sinhala. We show that when fine-tuned, these pre-trained language models set a very strong baseline for Sinhala text classification and are robust in situations where labeled data is insufficient for fine-tuning. We further provide a set of recommendations for using pre-trained models for Sinhala text classification. We also introduce new annotated datasets useful for future research in Sinhala text classification and publicly release our pre-trained models.
Submission history
From: Vinura Dhananjaya [view email][v1] Tue, 16 Aug 2022 17:47:42 GMT (711kb)
[v2] Wed, 17 Aug 2022 17:08:09 GMT (711kb)
Link back to: arXiv, form interface, contact.