We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.HC

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Sound

Title: Synthesizing Speech from Intracranial Depth Electrodes using an Encoder-Decoder Framework

Abstract: Speech Neuroprostheses have the potential to enable communication for people with dysarthria or anarthria. Recent advances have demonstrated high-quality text decoding and speech synthesis from electrocorticographic grids placed on the cortical surface. Here, we investigate a less invasive measurement modality in three participants, namely stereotactic EEG (sEEG) that provides sparse sampling from multiple brain regions, including subcortical regions. To evaluate whether sEEG can also be used to synthesize high-quality audio from neural recordings, we employ a recurrent encoder-decoder model based on modern deep learning methods. We find that speech can indeed be reconstructed with correlations up to 0.8 from these minimally invasive recordings, despite limited amounts of training data.
Subjects: Sound (cs.SD); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
Cite as: arXiv:2111.01457 [cs.SD]
  (or arXiv:2111.01457v2 [cs.SD] for this version)

Submission history

From: Jonas Kohler [view email]
[v1] Tue, 2 Nov 2021 09:43:21 GMT (13973kb,D)
[v2] Tue, 5 Jul 2022 11:18:11 GMT (14482kb,D)

Link back to: arXiv, form interface, contact.