References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Modality-Guided Subnetwork for Salient Object Detection
(Submitted on 10 Oct 2021 (v1), last revised 25 Oct 2021 (this version, v2))
Abstract: Recent RGBD-based models for saliency detection have attracted research attention. The depth clues such as boundary clues, surface normal, shape attribute, etc., contribute to the identification of salient objects with complicated scenarios. However, most RGBD networks require multi-modalities from the input side and feed them separately through a two-stream design, which inevitably results in extra costs on depth sensors and computation. To tackle these inconveniences, we present in this paper a novel fusion design named modality-guided subnetwork (MGSnet). It has the following superior designs: 1) Our model works for both RGB and RGBD data, and dynamically estimating depth if not available. Taking the inner workings of depth-prediction networks into account, we propose to estimate the pseudo-geometry maps from RGB input - essentially mimicking the multi-modality input. 2) Our MGSnet for RGB SOD results in real-time inference but achieves state-of-the-art performance compared to other RGB models. 3) The flexible and lightweight design of MGS facilitates the integration into RGBD two-streaming models. The introduced fusion design enables a cross-modality interaction to enable further progress but with a minimal cost.
Submission history
From: Zongwei Wu [view email][v1] Sun, 10 Oct 2021 20:59:11 GMT (7035kb,D)
[v2] Mon, 25 Oct 2021 14:54:03 GMT (7035kb,D)
Link back to: arXiv, form interface, contact.