We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Moûsai: Text-to-Music Generation with Long-Context Latent Diffusion

Abstract: The recent surge in popularity of diffusion models for image generation has brought new attention to the potential of these models in other areas of media synthesis. One area that has yet to be fully explored is the application of diffusion models to music generation. Music generation requires to handle multiple aspects, including the temporal dimension, long-term structure, multiple layers of overlapping sounds, and nuances that only trained listeners can detect. In our work, we investigate the potential of diffusion models for text-conditional music generation. We develop a cascading latent diffusion approach that can generate multiple minutes of high-quality stereo music at 48kHz from textual descriptions. For each model, we make an effort to maintain reasonable inference speed, targeting real-time on a single consumer GPU. In addition to trained models, we provide a collection of open-source libraries with the hope of facilitating future work in the field.
We open-source the following: Music samples for this paper: this https URL; all music samples for all models: this https URL; and codes: this https URL
Comments: Music samples for this paper: this https URL; all music samples for all models: this https URL; and codes: this https URL
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Cite as: arXiv:2301.11757 [cs.CL]
  (or arXiv:2301.11757v2 [cs.CL] for this version)

Submission history

From: Zhijing Jin [view email]
[v1] Fri, 27 Jan 2023 14:52:53 GMT (3860kb,D)
[v2] Mon, 30 Jan 2023 17:53:24 GMT (3860kb,D)

Link back to: arXiv, form interface, contact.