We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CV

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computer Vision and Pattern Recognition

Title: Knowledge Distillation for Incremental Learning in Semantic Segmentation

Abstract: Deep learning architectures have shown remarkable results in scene understanding problems, however they exhibit a critical drop of performances when they are required to learn incrementally new tasks without forgetting old ones. This catastrophic forgetting phenomenon impacts on the deployment of artificial intelligence in real world scenarios where systems need to learn new and different representations over time. Current approaches for incremental learning deal only with image classification and object detection tasks, while in this work we formally introduce incremental learning for semantic segmentation. We tackle the problem applying various knowledge distillation techniques on the previous model. In this way, we retain the information about learned classes, whilst updating the current model to learn the new ones. We developed four main methodologies of knowledge distillation working on both output layers and internal feature representations. We do not store any image belonging to previous training stages and only the last model is used to preserve high accuracy on previously learned classes. Extensive experimental results on the Pascal VOC2012 and MSRC-v2 datasets show the effectiveness of the proposed approaches in several incremental learning scenarios.
Comments: Computer Vision and Image Understanding (CVIU), 2021. arXiv admin note: text overlap with arXiv:1907.13372
Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Image and Video Processing (eess.IV)
Cite as: arXiv:1911.03462 [cs.CV]
  (or arXiv:1911.03462v4 [cs.CV] for this version)

Submission history

From: Umberto Michieli [view email]
[v1] Fri, 8 Nov 2019 08:17:03 GMT (4855kb,D)
[v2] Sun, 26 Jan 2020 11:29:10 GMT (7628kb,D)
[v3] Tue, 4 Feb 2020 17:53:16 GMT (7628kb,D)
[v4] Wed, 20 Jan 2021 20:03:16 GMT (9280kb,D)

Link back to: arXiv, form interface, contact.