Current browse context:
cs.CV
Change to browse by:
References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Knowledge Distillation for Incremental Learning in Semantic Segmentation
(Submitted on 8 Nov 2019 (this version), latest version 20 Jan 2021 (v4))
Abstract: Although deep learning architectures have shown remarkable results in scene understanding problems, they exhibit a critical drop of overall performance due to catastrophic forgetting when they are required to incrementally learn to recognize new classes without forgetting the old ones. This phenomenon impacts on the deployment of artificial intelligence in real world scenarios where systems need to learn new and different representations over time. Current approaches for incremental learning deal only with the image classification and object detection tasks. In this work we formally introduce the incremental learning problem for semantic segmentation. To avoid catastrophic forgetting we propose to distill the knowledge of the previous model to retain the information about previously learned classes, whilst updating the current model to learn the new ones. We developed three main methodologies of knowledge distillation working on both the output layers and the internal feature representations. Furthermore, differently from other recent frameworks, we do not store any image belonging to the previous training stages while only the last model is used to preserve high accuracy on previously learned classes. Extensive results were conducted on the Pascal VOC2012 dataset and show the effectiveness of the proposed approaches in different incremental learning scenarios.
Submission history
From: Umberto Michieli [view email][v1] Fri, 8 Nov 2019 08:17:03 GMT (4855kb,D)
[v2] Sun, 26 Jan 2020 11:29:10 GMT (7628kb,D)
[v3] Tue, 4 Feb 2020 17:53:16 GMT (7628kb,D)
[v4] Wed, 20 Jan 2021 20:03:16 GMT (9280kb,D)
Link back to: arXiv, form interface, contact.