We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.HC

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Human-Computer Interaction

Title: Latent Mappings: Generating Open-Ended Expressive Mappings Using Variational Autoencoders

Abstract: In many contexts, creating mappings for gestural interactions can form part of an artistic process. Creators seeking a mapping that is expressive, novel, and affords them a sense of authorship may not know how to program it up in a signal processing patch. Tools like Wekinator and MIMIC allow creators to use supervised machine learning to learn mappings from example input/output pairings. However, a creator may know a good mapping when they encounter it yet start with little sense of what the inputs or outputs should be. We call this an open-ended mapping process. Addressing this need, we introduce the latent mapping, which leverages the latent space of an unsupervised machine learning algorithm such as a Variational Autoencoder trained on a corpus of unlabelled gestural data from the creator. We illustrate it with Sonified Body, a system mapping full-body movement to sound which we explore in a residency with three dancers.
Comments: Published at the International Conference on New Interfaces for Musical Expression, June 2021. 3000 word short paper. 5 figures plus video which may be seen at this https URL
Subjects: Human-Computer Interaction (cs.HC); Multimedia (cs.MM)
Cite as: arXiv:2106.08867 [cs.HC]
  (or arXiv:2106.08867v1 [cs.HC] for this version)

Submission history

From: Tim Murray-Browne [view email]
[v1] Wed, 16 Jun 2021 15:40:53 GMT (478kb)

Link back to: arXiv, form interface, contact.