References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Fully Transformer Networks for Semantic Image Segmentation
(Submitted on 8 Jun 2021 (v1), last revised 28 Dec 2021 (this version, v3))
Abstract: Transformers have shown impressive performance in various natural language processing and computer vision tasks, due to the capability of modeling long-range dependencies. Recent progress has demonstrated that combining such Transformers with CNN-based semantic image segmentation models is very promising. However, it is not well studied yet on how well a pure Transformer based approach can achieve for image segmentation. In this work, we explore a novel framework for semantic image segmentation, which is encoder-decoder based Fully Transformer Networks (FTN). Specifically, we first propose a Pyramid Group Transformer (PGT) as the encoder for progressively learning hierarchical features, meanwhile reducing the computation complexity of the standard Visual Transformer (ViT). Then, we propose a Feature Pyramid Transformer (FPT) to fuse semantic-level and spatial-level information from multiple levels of the PGT encoder for semantic image segmentation. Surprisingly, this simple baseline can achieve better results on multiple challenging semantic segmentation and face parsing benchmarks, including PASCAL Context, ADE20K, COCOStuff, and CelebAMask-HQ. The source code will be released on this https URL
Submission history
From: Sitong Wu [view email][v1] Tue, 8 Jun 2021 05:15:28 GMT (11695kb,D)
[v2] Thu, 26 Aug 2021 11:13:35 GMT (5847kb,D)
[v3] Tue, 28 Dec 2021 05:50:43 GMT (9661kb,D)
Link back to: arXiv, form interface, contact.