References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: SparseFormer: Attention-based Depth Completion Network
(Submitted on 9 Jun 2022)
Abstract: Most pipelines for Augmented and Virtual Reality estimate the ego-motion of the camera by creating a map of sparse 3D landmarks. In this paper, we tackle the problem of depth completion, that is, densifying this sparse 3D map using RGB images as guidance. This remains a challenging problem due to the low density, non-uniform and outlier-prone 3D landmarks produced by SfM and SLAM pipelines. We introduce a transformer block, SparseFormer, that fuses 3D landmarks with deep visual features to produce dense depth. The SparseFormer has a global receptive field, making the module especially effective for depth completion with low-density and non-uniform landmarks. To address the issue of depth outliers among the 3D landmarks, we introduce a trainable refinement module that filters outliers through attention between the sparse landmarks.
Submission history
From: Frederik Warburg [view email][v1] Thu, 9 Jun 2022 15:08:24 GMT (31987kb,D)
Link back to: arXiv, form interface, contact.