References & Citations
Computer Science > Computation and Language
Title: Morphosyntactic Tagging with Pre-trained Language Models for Arabic and its Dialects
(Submitted on 13 Oct 2021 (v1), last revised 21 Mar 2022 (this version, v2))
Abstract: We present state-of-the-art results on morphosyntactic tagging across different varieties of Arabic using fine-tuned pre-trained transformer language models. Our models consistently outperform existing systems in Modern Standard Arabic and all the Arabic dialects we study, achieving 2.6% absolute improvement over the previous state-of-the-art in Modern Standard Arabic, 2.8% in Gulf, 1.6% in Egyptian, and 8.3% in Levantine. We explore different training setups for fine-tuning pre-trained transformer language models, including training data size, the use of external linguistic resources, and the use of annotated data from other dialects in a low-resource scenario. Our results show that strategic fine-tuning using datasets from other high-resource dialects is beneficial for a low-resource dialect. Additionally, we show that high-quality morphological analyzers as external linguistic resources are beneficial especially in low-resource settings.
Submission history
From: Go Inoue [view email][v1] Wed, 13 Oct 2021 16:43:44 GMT (764kb,D)
[v2] Mon, 21 Mar 2022 08:04:25 GMT (536kb,D)
Link back to: arXiv, form interface, contact.