Logarithmic Continual Learning
  We introduce a neural network architecture that logarithmically reduces the
number of self-rehearsal steps in the generative rehearsal of continually
learned models. In continual learning (CL), training samples come in subsequent
tasks, and the trained model can access only a single task at a time. To replay
previous samples, contemporary CL methods bootstrap generative models and train
them recursively with a combination of current and regenerated past data. This
recurrence leads to superfluous computations as the same past samples are
regenerated after each task, and the reconstruction quality successively
degrades. In this work, we address these limitations and propose a new
generative rehearsal architecture that requires at most logarithmic number of
retraining for each sample. Our approach leverages allocation of past data in
a~set of generative models such that most of them do not require retraining
after a~task. The experimental evaluation of our logarithmic continual learning
approach shows the superiority of our method with respect to the
state-of-the-art generative rehearsal methods.
