Ex-Model: Continual Learning from a Stream of Trained Models
  Learning continually from non-stationary data streams is a challenging
research topic of growing popularity in the last few years. Being able to
learn, adapt, and generalize continually in an efficient, effective, and
scalable way is fundamental for a sustainable development of Artificial
Intelligent systems. However, an agent-centric view of continual learning
requires learning directly from raw data, which limits the interaction between
independent agents, the efficiency, and the privacy of current approaches.
Instead, we argue that continual learning systems should exploit the
availability of compressed information in the form of trained models. In this
paper, we introduce and formalize a new paradigm named "Ex-Model Continual
Learning" (ExML), where an agent learns from a sequence of previously trained
models instead of raw data. We further contribute with three ex-model continual
learning algorithms and an empirical setting comprising three datasets (MNIST,
CIFAR-10 and CORe50), and eight scenarios, where the proposed algorithms are
extensively tested. Finally, we highlight the peculiarities of the ex-model
paradigm and we point out interesting future research directions.
