TRGP: Trust Region Gradient Projection for Continual Learning
  Catastrophic forgetting is one of the major challenges in continual learning.
To address this issue, some existing methods put restrictive constraints on the
optimization space of the new task for minimizing the interference to old
tasks. However, this may lead to unsatisfactory performance for the new task,
especially when the new task is strongly correlated with old tasks. To tackle
this challenge, we propose Trust Region Gradient Projection (TRGP) for
continual learning to facilitate the forward knowledge transfer based on an
efficient characterization of task correlation. Particularly, we introduce a
notion of `trust region' to select the most related old tasks for the new task
in a layer-wise and single-shot manner, using the norm of gradient projection
onto the subspace spanned by task inputs. Then, a scaled weight projection is
proposed to cleverly reuse the frozen weights of the selected old tasks in the
trust region through a layer-wise scaling matrix. By jointly optimizing the
scaling matrices and the model, where the model is updated along the directions
orthogonal to the subspaces of old tasks, TRGP can effectively prompt knowledge
transfer without forgetting. Extensive experiments show that our approach
achieves significant improvement over related state-of-the-art methods.
