We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CV

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computer Vision and Pattern Recognition

Title: Knowledge as Priors: Cross-Modal Knowledge Generalization for Datasets without Superior Knowledge

Abstract: Cross-modal knowledge distillation deals with transferring knowledge from a model trained with superior modalities (Teacher) to another model trained with weak modalities (Student). Existing approaches require paired training examples exist in both modalities. However, accessing the data from superior modalities may not always be feasible. For example, in the case of 3D hand pose estimation, depth maps, point clouds, or stereo images usually capture better hand structures than RGB images, but most of them are expensive to be collected. In this paper, we propose a novel scheme to train the Student in a Target dataset where the Teacher is unavailable. Our key idea is to generalize the distilled cross-modal knowledge learned from a Source dataset, which contains paired examples from both modalities, to the Target dataset by modeling knowledge as priors on parameters of the Student. We name our method "Cross-Modal Knowledge Generalization" and demonstrate that our scheme results in competitive performance for 3D hand pose estimation on standard benchmark datasets.
Comments: In CVPR 2020. (15 pages including supplementary material)
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:2004.00176 [cs.CV]
  (or arXiv:2004.00176v1 [cs.CV] for this version)

Submission history

From: Long Zhao [view email]
[v1] Wed, 1 Apr 2020 00:28:15 GMT (5111kb,D)

Link back to: arXiv, form interface, contact.