We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Non-Autoregressive Sign Language Production via Knowledge Distillation

Abstract: Sign Language Production (SLP) aims to translate expressions in spoken language into corresponding ones in sign language, such as skeleton-based sign poses or videos. Existing SLP models are either AutoRegressive (AR) or Non-Autoregressive (NAR). However, AR-SLP models suffer from regression to the mean and error propagation during decoding. NSLP-G, a NAR-based model, resolves these issues to some extent but engenders other problems. For example, it does not consider target sign lengths and suffers from false decoding initiation. We propose a novel NAR-SLP model via Knowledge Distillation (KD) to address these problems. First, we devise a length regulator to predict the end of the generated sign pose sequence. We then adopt KD, which distills spatial-linguistic features from a pre-trained pose encoder to alleviate false decoding initiation. Extensive experiments show that the proposed approach significantly outperforms existing SLP models in both Frechet Gesture Distance and Back-Translation evaluation.
Comments: 10 pages, 4 figures, 3 tables, submitted to ECCV2023
Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL)
Cite as: arXiv:2208.06183 [cs.LG]
  (or arXiv:2208.06183v1 [cs.LG] for this version)

Submission history

From: Eui Jun Hwang [view email]
[v1] Fri, 12 Aug 2022 09:17:11 GMT (2981kb,D)

Link back to: arXiv, form interface, contact.