References & Citations
Computer Science > Computation and Language
Title: Emotional Conversation Generation with Heterogeneous Graph Neural Network
(Submitted on 9 Dec 2020 (v1), last revised 30 Mar 2022 (this version, v3))
Abstract: The successful emotional conversation system depends on sufficient perception and appropriate expression of emotions. In a real-life conversation, humans firstly instinctively perceive emotions from multi-source information, including the emotion flow hidden in dialogue history, facial expressions, audio, and personalities of speakers. Then, they convey suitable emotions according to their personalities, but these multiple types of information are insufficiently exploited in emotional conversation fields. To address this issue, in this paper, we propose a heterogeneous graph-based model for emotional conversation generation. Firstly, we design a Heterogeneous Graph-Based Encoder to represent the conversation content (i.e., the dialogue history, its emotion flow, facial expressions, audio, and speakers' personalities) with a heterogeneous graph neural network, and then predict suitable emotions for feedback. Secondly, we employ an Emotion-Personality-Aware Decoder to generate a response relevant to the conversation context as well as with appropriate emotions, through taking the encoded graph representations, the predicted emotions by the encoder and the personality of the current speaker as inputs. Experiments on both automatic and human evaluation show that our method can effectively perceive emotions from multi-source knowledge and generate a satisfactory response. Furthermore, based on the up-to-date text generator BART, our model still can achieve consistent improvement, which significantly outperforms some existing state-of-the-art models.
Submission history
From: Yunlong Liang [view email][v1] Wed, 9 Dec 2020 06:09:31 GMT (4748kb,D)
[v2] Wed, 15 Sep 2021 09:28:06 GMT (5278kb,D)
[v3] Wed, 30 Mar 2022 15:29:53 GMT (1264kb,D)
Link back to: arXiv, form interface, contact.