References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: The Missing Link: Finding label relations across datasets
(Submitted on 9 Jun 2022 (v1), last revised 9 Aug 2022 (this version, v2))
Abstract: Computer vision is driven by the many datasets available for training or evaluating novel methods. However, each dataset has a different set of class labels, visual definition of classes, images following a specific distribution, annotation protocols, etc. In this paper we explore the automatic discovery of visual-semantic relations between labels across datasets. We aim to understand how instances of a certain class in a dataset relate to the instances of another class in another dataset. Are they in an identity, parent/child, overlap relation? Or is there no link between them at all? To find relations between labels across datasets, we propose methods based on language, on vision, and on their combination. We show that we can effectively discover label relations across datasets, as well as their type. We apply our method to four applications: understand label relations, identify missing aspects, increase label specificity, and predict transfer learning gains. We conclude that label relations cannot be established by looking at the names of classes alone, as they depend strongly on how each of the datasets was constructed.
Submission history
From: Jasper Uijlings [view email][v1] Thu, 9 Jun 2022 12:25:25 GMT (13119kb,D)
[v2] Tue, 9 Aug 2022 13:30:40 GMT (7884kb,D)
Link back to: arXiv, form interface, contact.