References & Citations
Electrical Engineering and Systems Science > Audio and Speech Processing
Title: Analyzing the Robustness of Unsupervised Speech Recognition
(Submitted on 7 Oct 2021 (v1), last revised 26 Apr 2022 (this version, v5))
Abstract: Unsupervised speech recognition (unsupervised ASR) aims to learn the ASR system with non-parallel speech and text corpus only. Wav2vec-U has shown promising results in unsupervised ASR by self-supervised speech representations coupled with Generative Adversarial Network (GAN) training, but the robustness of the unsupervised ASR framework is unknown. In this work, we further analyze the training robustness of unsupervised ASR on the domain mismatch scenarios in which the domains of unpaired speech and text are different. Three domain mismatch scenarios include: (1) using speech and text from different datasets, (2) utilizing noisy/spontaneous speech, and (3) adjusting the amount of speech and text data. We also quantify the degree of the domain mismatch by calculating the JS-divergence of phoneme n-gram between the transcription of speech and text. This metric correlates with the performance highly. Experimental results show that domain mismatch leads to inferior performance, but a self-supervised model pre-trained on the targeted speech domain can extract better representation to alleviate the performance drop.
Submission history
From: Guan-Ting Lin [view email][v1] Thu, 7 Oct 2021 14:46:10 GMT (42kb,D)
[v2] Fri, 8 Oct 2021 03:20:11 GMT (42kb,D)
[v3] Tue, 12 Oct 2021 05:10:24 GMT (42kb,D)
[v4] Sat, 12 Feb 2022 12:52:37 GMT (109kb,D)
[v5] Tue, 26 Apr 2022 11:51:42 GMT (110kb,D)
Link back to: arXiv, form interface, contact.