Current browse context:
stat.ML
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Speech-driven facial animation using polynomial fusion of features
(Submitted on 12 Dec 2019 (v1), last revised 19 Feb 2020 (this version, v2))
Abstract: Speech-driven facial animation involves using a speech signal to generate realistic videos of talking faces. Recent deep learning approaches to facial synthesis rely on extracting low-dimensional representations and concatenating them, followed by a decoding step of the concatenated vector. This accounts for only first-order interactions of the features and ignores higher-order interactions. In this paper we propose a polynomial fusion layer that models the joint representation of the encodings by a higher-order polynomial, with the parameters modelled by a tensor decomposition. We demonstrate the suitability of this approach through experiments on generated videos evaluated on a range of metrics on video quality, audiovisual synchronisation and generation of blinks.
Submission history
From: Triantafyllos Kefalas [view email][v1] Thu, 12 Dec 2019 08:46:57 GMT (698kb,D)
[v2] Wed, 19 Feb 2020 14:36:16 GMT (694kb,D)
Link back to: arXiv, form interface, contact.