Current browse context:
eess.AS
Change to browse by:
References & Citations
Electrical Engineering and Systems Science > Audio and Speech Processing
Title: Multi-task Voice Activated Framework using Self-supervised Learning
(Submitted on 3 Oct 2021 (v1), last revised 19 Mar 2022 (this version, v3))
Abstract: Self-supervised learning methods such as wav2vec 2.0 have shown promising results in learning speech representations from unlabelled and untranscribed speech data that are useful for speech recognition. Since these representations are learned without any task-specific supervision, they can also be useful for other voice-activated tasks like speaker verification, keyword spotting, emotion classification etc. In our work, we propose a general purpose framework for adapting a pre-trained wav2vec 2.0 model for different voice-activated tasks. We develop downstream network architectures that operate on the contextualized speech representations of wav2vec 2.0 to adapt the representations for solving a given task. Finally, we extend our framework to perform multi-task learning by jointly optimizing the network parameters on multiple voice activated tasks using a shared transformer backbone. Both of our single and multi-task frameworks achieve state-of-the-art results in speaker verification and keyword spotting benchmarks. Our best performing models achieve 1.98% and 3.15% EER on VoxCeleb1 test set when trained on VoxCeleb2 and VoxCeleb1 respectively, and 98.23% accuracy on Google Speech Commands v1.0 keyword spotting dataset.
Submission history
From: Shehzeen Hussain [view email][v1] Sun, 3 Oct 2021 19:28:57 GMT (2527kb,D)
[v2] Tue, 12 Oct 2021 07:13:30 GMT (2527kb,D)
[v3] Sat, 19 Mar 2022 04:28:49 GMT (2527kb,D)
Link back to: arXiv, form interface, contact.