We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CV

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computer Vision and Pattern Recognition

Title: PointCMC: Cross-Modal Multi-Scale Correspondences Learning for Point Cloud Understanding

Abstract: Some self-supervised cross-modal learning approaches have recently demonstrated the potential of image signals for enhancing point cloud representation. However, it remains a question on how to directly model cross-modal local and global correspondences in a self-supervised fashion. To solve it, we proposed PointCMC, a novel cross-modal method to model multi-scale correspondences across modalities for self-supervised point cloud representation learning. In particular, PointCMC is composed of: (1) a local-to-local (L2L) module that learns local correspondences through optimized cross-modal local geometric features, (2) a local-to-global (L2G) module that aims to learn the correspondences between local and global features across modalities via local-global discrimination, and (3) a global-to-global (G2G) module, which leverages auxiliary global contrastive loss between the point cloud and image to learn high-level semantic correspondences. Extensive experiment results show that our approach outperforms existing state-of-the-art methods in various downstream tasks such as 3D object classification and segmentation. Code will be made publicly available upon acceptance.
Comments: In order to revise the paper
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:2211.12032 [cs.CV]
  (or arXiv:2211.12032v2 [cs.CV] for this version)

Submission history

From: Honggu Zhou [view email]
[v1] Tue, 22 Nov 2022 06:08:43 GMT (1606kb,D)
[v2] Wed, 23 Nov 2022 15:06:57 GMT (0kb,I)

Link back to: arXiv, form interface, contact.