Addressing the Stability-Plasticity Dilemma via Knowledge-Aware
  Continual Learning
  Current methods in continual learning (CL) tend to focus on alleviating
catastrophic forgetting of previous tasks. This hinders balancing other CL
desiderata such as forward transfer, and memory and computation efficiency. CL
desiderata become much more competing when a model operates on
class-Incremental Learning (class-IL) without accessing past data since it is
prone to ambiguities between old and new classes. In this paper, we present
Knowledge-Aware coNtinual learner (KAN) that attempts to study the
stability-plasticity dilemma to balance CL desiderata in class-IL. In
particular, KAN is a new task-specific components method based on dynamic
sparse training that introduces reusability and selective transfer of past
knowledge in this class of methods. KAN selectively reuses relevant knowledge
while addressing class ambiguity, preserves old knowledge, and utilizes the
model capacity efficiently. Experiments show the effectiveness of KAN in
providing models with multiple CL desirable properties, outperforming
state-of-the-art methods on various challenging benchmarks.
