Current browse context:
cs.CL
Change to browse by:
References & Citations
Computer Science > Computation and Language
Title: ASR-GLUE: A New Multi-task Benchmark for ASR-Robust Natural Language Understanding
(Submitted on 30 Aug 2021 (v1), last revised 16 Mar 2022 (this version, v2))
Abstract: Language understanding in speech-based systems have attracted much attention in recent years with the growing demand for voice interface applications. However, the robustness of natural language understanding (NLU) systems to errors introduced by automatic speech recognition (ASR) is under-examined. %To facilitate the research on ASR-robust general language understanding, In this paper, we propose ASR-GLUE benchmark, a new collection of 6 different NLU tasks for evaluating the performance of models under ASR error across 3 different levels of background noise and 6 speakers with various voice characteristics. Based on the proposed benchmark, we systematically investigate the effect of ASR error on NLU tasks in terms of noise intensity, error type and speaker variants. We further purpose two ways, correction-based method and data augmentation-based method to improve robustness of the NLU systems. Extensive experimental results and analysises show that the proposed methods are effective to some extent, but still far from human performance, demonstrating that NLU under ASR error is still very challenging and requires further research.
Submission history
From: Jianwei Yu [view email][v1] Mon, 30 Aug 2021 08:11:39 GMT (1254kb,D)
[v2] Wed, 16 Mar 2022 16:24:41 GMT (1246kb,D)
Link back to: arXiv, form interface, contact.