Continual Learning with Echo State Networks
  Continual Learning (CL) refers to a learning setup where data is non
stationary and the model has to learn without forgetting existing knowledge.
The study of CL for sequential patterns revolves around trained recurrent
networks. In this work, instead, we introduce CL in the context of Echo State
Networks (ESNs), where the recurrent component is kept fixed. We provide the
first evaluation of catastrophic forgetting in ESNs and we highlight the
benefits in using CL strategies which are not applicable to trained recurrent
models. Our results confirm the ESN as a promising model for CL and open to its
use in streaming scenarios.
