Current browse context:
cs.CV
Change to browse by:
References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection
(Submitted on 1 Jun 2021 (v1), last revised 27 Oct 2021 (this version, v3))
Abstract: Can Transformer perform 2D object- and region-level recognition from a pure sequence-to-sequence perspective with minimal knowledge about the 2D spatial structure? To answer this question, we present You Only Look at One Sequence (YOLOS), a series of object detection models based on the vanilla Vision Transformer with the fewest possible modifications, region priors, as well as inductive biases of the target task. We find that YOLOS pre-trained on the mid-sized ImageNet-1k dataset only can already achieve quite competitive performance on the challenging COCO object detection benchmark, e.g., YOLOS-Base directly adopted from BERT-Base architecture can obtain 42.0 box AP on COCO val. We also discuss the impacts as well as limitations of current pre-train schemes and model scaling strategies for Transformer in vision through YOLOS. Code and pre-trained models are available at this https URL
Submission history
From: Yuxin Fang [view email][v1] Tue, 1 Jun 2021 17:54:09 GMT (2914kb,D)
[v2] Mon, 21 Jun 2021 02:28:30 GMT (16431kb,D)
[v3] Wed, 27 Oct 2021 02:14:12 GMT (16435kb,D)
Link back to: arXiv, form interface, contact.