Target Layer Regularization for Continual Learning Using Cramer-Wold
  Generator
  We propose an effective regularization strategy (CW-TaLaR) for solving
continual learning problems. It uses a penalizing term expressed by the
Cramer-Wold distance between two probability distributions defined on a target
layer of an underlying neural network that is shared by all tasks, and the
simple architecture of the Cramer-Wold generator for modeling output data
representation. Our strategy preserves target layer distribution while learning
a new task but does not require remembering previous tasks' datasets. We
perform experiments involving several common supervised frameworks, which prove
the competitiveness of the CW-TaLaR method in comparison to a few existing
state-of-the-art continual learning models.
