References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Adversarial Semantic Hallucination for Domain Generalized Semantic Segmentation
(Submitted on 8 Jun 2021 (v1), last revised 26 Oct 2021 (this version, v6))
Abstract: Convolutional neural networks typically perform poorly when the test (target domain) and training (source domain) data have significantly different distributions. While this problem can be mitigated by using the target domain data to align the source and target domain feature representations, the target domain data may be unavailable due to privacy concerns. Consequently, there is a need for methods that generalize well despite restricted access to target domain data during training. In this work, we propose an adversarial semantic hallucination approach (ASH), which combines a class-conditioned hallucination module and a semantic segmentation module. Since the segmentation performance varies across different classes, we design a semantic-conditioned style hallucination module to generate affine transformation parameters from semantic information in the segmentation probability maps of the source domain image. Unlike previous adaptation approaches, which treat all classes equally, ASH considers the class-wise differences. The segmentation module and the hallucination module compete adversarially, with the hallucination module generating increasingly "difficult" stylized images to challenge the segmentation module. In response, the segmentation module improves as it is trained with generated samples at an appropriate class-wise difficulty level. Our results on the Cityscapes and Mapillary benchmark datasets show that our method is competitive with state of the art work. Code is made available at this https URL
Submission history
From: Gabriel Tjio [view email][v1] Tue, 8 Jun 2021 07:07:45 GMT (33895kb,D)
[v2] Thu, 8 Jul 2021 04:05:26 GMT (33897kb,D)
[v3] Thu, 16 Sep 2021 08:01:26 GMT (5877kb,D)
[v4] Wed, 6 Oct 2021 02:11:13 GMT (5877kb,D)
[v5] Mon, 25 Oct 2021 15:22:36 GMT (36739kb)
[v6] Tue, 26 Oct 2021 14:20:35 GMT (37272kb)
Link back to: arXiv, form interface, contact.