Knowledge Distillation for Incremental Learning in Semantic Segmentation
  Deep learning architectures have shown remarkable results in scene
understanding problems, however they exhibit a critical drop of performances
when they are required to learn incrementally new tasks without forgetting old
ones. This catastrophic forgetting phenomenon impacts on the deployment of
artificial intelligence in real world scenarios where systems need to learn new
and different representations over time. Current approaches for incremental
learning deal only with image classification and object detection tasks, while
in this work we formally introduce incremental learning for semantic
segmentation. We tackle the problem applying various knowledge distillation
techniques on the previous model. In this way, we retain the information about
learned classes, whilst updating the current model to learn the new ones. We
developed four main methodologies of knowledge distillation working on both
output layers and internal feature representations. We do not store any image
belonging to previous training stages and only the last model is used to
preserve high accuracy on previously learned classes. Extensive experimental
results on the Pascal VOC2012 and MSRC-v2 datasets show the effectiveness of
the proposed approaches in several incremental learning scenarios.
