Current browse context:
cs.SD
Change to browse by:
References & Citations
Computer Science > Sound
Title: Voice Reconstruction from Silent Speech with a Sequence-to-Sequence Model
(Submitted on 31 Jul 2021 (this version), latest version 1 Jun 2022 (v3))
Abstract: Silent Speech Decoding (SSD) based on Surface electromyography (sEMG) has become a prevalent task in recent years. Though revolutions have been proposed to decode sEMG to audio successfully, some problems still remain. In this paper, we propose an optimized sequence-to-sequence (Seq2Seq) approach to synthesize voice from subvocal sEMG. Both subvocal and vocal sEMG are collected and preprocessed to provide data information. Then, we extract durations from the alignment between subvocal and vocal signals to regulate the subvocal sEMG following audio length. Besides, we use phoneme classification and vocal sEMG reconstruction modules to improve the model performance. Finally, experiments on a Mandarin speaker dataset, which consists of 6.49 hours of data, demonstrate that the proposed model improves the mapping accuracy and voice quality of reconstructed voice.
Submission history
From: Huiyan Li [view email][v1] Sat, 31 Jul 2021 09:28:06 GMT (2474kb,D)
[v2] Mon, 28 Mar 2022 02:51:51 GMT (11936kb)
[v3] Wed, 1 Jun 2022 14:52:15 GMT (11722kb)
Link back to: arXiv, form interface, contact.