Weight Friction: A Simple Method to Overcome Catastrophic Forgetting and
  Enable Continual Learning
  In recent years, deep neural networks have found success in replicating
human-level cognitive skills, yet they suffer from several major obstacles. One
significant limitation is the inability to learn new tasks without forgetting
previously learned tasks, a shortcoming known as catastrophic forgetting. In
this research, we propose a simple method to overcome catastrophic forgetting
and enable continual learning in neural networks. We draw inspiration from
principles in neurology and physics to develop the concept of weight friction.
Weight friction operates by a modification to the update rule in the gradient
descent optimization method. It converges at a rate comparable to that of the
stochastic gradient descent algorithm and can operate over multiple task
domains. It performs comparably to current methods while offering improvements
in computation and memory efficiency.
