References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: UniT: Unified Knowledge Transfer for Any-shot Object Detection and Segmentation
(Submitted on 12 Jun 2020 (v1), last revised 3 Mar 2021 (this version, v3))
Abstract: Methods for object detection and segmentation rely on large scale instance-level annotations for training, which are difficult and time-consuming to collect. Efforts to alleviate this look at varying degrees and quality of supervision. Weakly-supervised approaches draw on image-level labels to build detectors/segmentors, while zero/few-shot methods assume abundant instance-level data for a set of base classes, and none to a few examples for novel classes. This taxonomy has largely siloed algorithmic designs. In this work, we aim to bridge this divide by proposing an intuitive and unified semi-supervised model that is applicable to a range of supervision: from zero to a few instance-level samples per novel class. For base classes, our model learns a mapping from weakly-supervised to fully-supervised detectors/segmentors. By learning and leveraging visual and lingual similarities between the novel and base classes, we transfer those mappings to obtain detectors/segmentors for novel classes; refining them with a few novel class instance-level annotated samples, if available. The overall model is end-to-end trainable and highly flexible. Through extensive experiments on MS-COCO and Pascal VOC benchmark datasets we show improved performance in a variety of settings.
Submission history
From: Siddhesh Khandelwal [view email][v1] Fri, 12 Jun 2020 22:45:47 GMT (7950kb,D)
[v2] Wed, 9 Dec 2020 17:38:45 GMT (8054kb,D)
[v3] Wed, 3 Mar 2021 20:34:48 GMT (6536kb,D)
Link back to: arXiv, form interface, contact.