SSUL: Semantic Segmentation with Unknown Label for Exemplar-based
  Class-Incremental Learning
  This paper introduces a solid state-of-the-art baseline for a
class-incremental semantic segmentation (CISS) problem. While the recent CISS
algorithms utilize variants of the knowledge distillation (KD) technique to
tackle the problem, they failed to fully address the critical challenges in
CISS causing the catastrophic forgetting; the semantic drift of the background
class and the multi-label prediction issue. To better address these challenges,
we propose a new method, dubbed SSUL-M (Semantic Segmentation with Unknown
Label with Memory), by carefully combining techniques tailored for semantic
segmentation. Specifically, we claim three main contributions. (1) defining
unknown classes within the background class to help to learn future classes
(help plasticity), (2) freezing backbone network and past classifiers with
binary cross-entropy loss and pseudo-labeling to overcome catastrophic
forgetting (help stability), and (3) utilizing tiny exemplar memory for the
first time in CISS to improve both plasticity and stability. The extensively
conducted experiments show the effectiveness of our method, achieving
significantly better performance than the recent state-of-the-art baselines on
the standard benchmark datasets. Furthermore, we justify our contributions with
thorough ablation analyses and discuss different natures of the CISS problem
compared to the traditional class-incremental learning targeting
classification. The official code is available at
https://github.com/clovaai/SSUL.
