We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Speech Emotion Recognition using Multi-task learning and a multimodal dynamic fusion network

Abstract: Emotion Recognition (ER) aims to classify human utterances into different emotion categories. Based on early-fusion and self-attention-based multimodal interaction between text and acoustic modalities, in this paper, we propose MMER, a multimodal multitask learning approach for ER from individual utterances in isolation. Our proposed MMER leverages a multimodal dynamic fusion network that adds minimal parameters over an existing speech encoder to leverage the semantic and syntactic properties hidden in text. Experiments on the IEMOCAP benchmark show that our proposed model achieves state-of-the-art performance. In addition, strong baselines and ablation studies prove the effectiveness of our proposed approach. We make our code publicly available on GitHub.
Comments: 6 + 2 pages
Subjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Cite as: arXiv:2203.16794 [cs.CL]
  (or arXiv:2203.16794v4 [cs.CL] for this version)

Submission history

From: Sreyan Ghosh [view email]
[v1] Thu, 31 Mar 2022 04:51:32 GMT (312kb,D)
[v2] Fri, 1 Apr 2022 04:39:53 GMT (312kb,D)
[v3] Thu, 18 Aug 2022 15:12:39 GMT (312kb,D)
[v4] Mon, 31 Oct 2022 21:51:49 GMT (1446kb,D)

Link back to: arXiv, form interface, contact.