Uncertainty-based Continual Learning with Adaptive Regularization
  We introduce a new neural network-based continual learning algorithm, dubbed
as Uncertainty-regularized Continual Learning (UCL), which builds on
traditional Bayesian online learning framework with variational inference. We
focus on two significant drawbacks of the recently proposed
regularization-based methods: a) considerable additional memory cost for
determining the per-weight regularization strengths and b) the absence of
gracefully forgetting scheme, which can prevent performance degradation in
learning new tasks. In this paper, we show UCL can solve these two problems by
introducing a fresh interpretation on the Kullback-Leibler (KL) divergence term
of the variational lower bound for Gaussian mean-field approximation. Based on
the interpretation, we propose the notion of node-wise uncertainty, which
drastically reduces the number of additional parameters for implementing
per-weight regularization. Moreover, we devise two additional regularization
terms that enforce stability by freezing important parameters for past tasks
and allow plasticity by controlling the actively learning parameters for a new
task. Through extensive experiments, we show UCL convincingly outperforms most
of recent state-of-the-art baselines not only on popular supervised learning
benchmarks, but also on challenging lifelong reinforcement learning tasks. The
source code of our algorithm is available at https://github.com/csm9493/UCL.
