Current browse context:
eess.AS
Change to browse by:
References & Citations
Electrical Engineering and Systems Science > Audio and Speech Processing
Title: Boosting Multi-Speaker Expressive Speech Synthesis with Semi-supervised Contrastive Learning
(Submitted on 26 Oct 2023 (v1), last revised 25 Apr 2024 (this version, v2))
Abstract: This paper aims to build a multi-speaker expressive TTS system, synthesizing a target speaker's speech with multiple styles and emotions. To this end, we propose a novel contrastive learning-based TTS approach to transfer style and emotion across speakers. Specifically, contrastive learning from different levels, i.e. utterance and category level, is leveraged to extract the disentangled style, emotion, and speaker representations from speech for style and emotion transfer. Furthermore, a semi-supervised training strategy is introduced to improve the data utilization efficiency by involving multi-domain data, including style-labeled data, emotion-labeled data, and abundant unlabeled data. To achieve expressive speech with diverse styles and emotions for a target speaker, the learned disentangled representations are integrated into an improved VITS model. Experiments on multi-domain data demonstrate the effectiveness of the proposed method.
Submission history
From: Xinfa Zhu [view email][v1] Thu, 26 Oct 2023 01:58:38 GMT (2171kb,D)
[v2] Thu, 25 Apr 2024 14:41:55 GMT (2660kb,D)
Link back to: arXiv, form interface, contact.