We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CV

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computer Vision and Pattern Recognition

Title: Indicative Image Retrieval: Turning Blackbox Learning into Grey

Authors: Xulu Zhang (1), Zhenqun Yang (2), Hao Tian (1), Qing Li (3), Xiaoyong Wei (1 and 3) ((1) Sichuan University, (2) Chinese University of Hong Kong, (3) Hong Kong Polytechnic Univeristy)
Abstract: Deep learning became the game changer for image retrieval soon after it was introduced. It promotes the feature extraction (by representation learning) as the core of image retrieval, with the relevance/matching evaluation being degenerated into simple similarity metrics. In many applications, we need the matching evidence to be indicated rather than just have the ranked list (e.g., the locations of the target proteins/cells/lesions in medical images). It is like the matched words need to be highlighted in search engines. However, this is not easy to implement without explicit relevance/matching modeling. The deep representation learning models are not feasible because of their blackbox nature. In this paper, we revisit the importance of relevance/matching modeling in deep learning era with an indicative retrieval setting. The study shows that it is possible to skip the representation learning and model the matching evidence directly. By removing the dependency on the pre-trained models, it has avoided a lot of related issues (e.g., the domain gap between classification and retrieval, the detail-diffusion caused by convolution, and so on). More importantly, the study demonstrates that the matching can be explicitly modeled and backtracked later for generating the matching evidence indications. It can improve the explainability of deep inference. Our method obtains a best performance in literature on both Oxford-5k and Paris-6k, and sets a new record of 97.77% on Oxford-5k (97.81% on Paris-6k) without extracting any deep features.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
Cite as: arXiv:2201.11898 [cs.CV]
  (or arXiv:2201.11898v2 [cs.CV] for this version)

Submission history

From: Xulu Zhang [view email]
[v1] Fri, 28 Jan 2022 02:21:09 GMT (1570kb,D)
[v2] Wed, 16 Mar 2022 09:29:23 GMT (1570kb,D)

Link back to: arXiv, form interface, contact.