Current browse context:
eess.AS
Change to browse by:
References & Citations
Electrical Engineering and Systems Science > Audio and Speech Processing
Title: VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition
(Submitted on 12 Sep 2022 (v1), last revised 4 Oct 2022 (this version, v2))
Abstract: This paper presents a novel streaming automatic speech recognition (ASR) framework for multi-talker overlapping speech captured by a distant microphone array with an arbitrary geometry. Our framework, named t-SOT-VA, capitalizes on independently developed two recent technologies; array-geometry-agnostic continuous speech separation, or VarArray, and streaming multi-talker ASR based on token-level serialized output training (t-SOT). To combine the best of both technologies, we newly design a t-SOT-based ASR model that generates a serialized multi-talker transcription based on two separated speech signals from VarArray. We also propose a pre-training scheme for such an ASR model where we simulate VarArray's output signals based on monaural single-talker ASR training data. Conversation transcription experiments using the AMI meeting corpus show that the system based on the proposed framework significantly outperforms conventional ones. Our system achieves the state-of-the-art word error rates of 13.7% and 15.5% for the AMI development and evaluation sets, respectively, in the multiple-distant-microphone setting while retaining the streaming inference capability.
Submission history
From: Naoyuki Kanda [view email][v1] Mon, 12 Sep 2022 01:22:04 GMT (344kb,D)
[v2] Tue, 4 Oct 2022 03:29:31 GMT (346kb,D)
Link back to: arXiv, form interface, contact.