We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Computer Science > Computation and Language

Title: Look Ma, Only 400 Samples! Revisiting the Effectiveness of Automatic N-Gram Rule Generation for Spelling Normalization in Filipino

Abstract: With 84.75 million Filipinos online, the ability for models to process online text is crucial for developing Filipino NLP applications. To this end, spelling correction is a crucial preprocessing step for downstream processing. However, the lack of data prevents the use of language models for this task. In this paper, we propose an N-Gram + Damerau Levenshtein distance model with automatic rule extraction. We train the model on 300 samples, and show that despite limited training data, it achieves good performance and outperforms other deep learning approaches in terms of accuracy and edit distance. Moreover, the model (1) requires little compute power, (2) trains in little time, thus allowing for retraining, and (3) is easily interpretable, allowing for direct troubleshooting, highlighting the success of traditional approaches over more complex deep learning models in settings where data is unavailable.
Comments: 4 pages, 1 figure, Presented at EMNLP 2022 Third Workshop on Simple and Efficient Natural Language Processing
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2210.02675 [cs.CL]
  (or arXiv:2210.02675v2 [cs.CL] for this version)

Submission history

From: Lorenzo Jaime Flores [view email]
[v1] Thu, 6 Oct 2022 04:41:26 GMT (7380kb,D)
[v2] Sat, 5 Nov 2022 15:02:12 GMT (7379kb,D)

Link back to: arXiv, form interface, contact.