We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.SD

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Sound

Title: Zero-Shot Voice Conditioning for Denoising Diffusion TTS Models

Abstract: We present a novel way of conditioning a pretrained denoising diffusion speech model to produce speech in the voice of a novel person unseen during training. The method requires a short (~3 seconds) sample from the target person, and generation is steered at inference time, without any training steps. At the heart of the method lies a sampling process that combines the estimation of the denoising model with a low-pass version of the new speaker's sample. The objective and subjective evaluations show that our sampling method can generate a voice similar to that of the target speaker in terms of frequency, with an accuracy comparable to state-of-the-art methods, and without training.
Comments: Accepted to Interspeech 2022
Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS); Signal Processing (eess.SP)
Cite as: arXiv:2206.02246 [cs.SD]
  (or arXiv:2206.02246v2 [cs.SD] for this version)

Submission history

From: Eliya Nachmani [view email]
[v1] Sun, 5 Jun 2022 19:45:29 GMT (1337kb,D)
[v2] Wed, 22 Jun 2022 09:22:35 GMT (1336kb,D)

Link back to: arXiv, form interface, contact.