S-TRIGGER: Continual State Representation Learning via Self-Triggered
  Generative Replay
  We consider the problem of building a state representation model for control,
in a continual learning setting. As the environment changes, the aim is to
efficiently compress the sensory state's information without losing past
knowledge, and then use Reinforcement Learning on the resulting features for
efficient policy learning. To this end, we propose S-TRIGGER, a general method
for Continual State Representation Learning applicable to Variational
Auto-Encoders and its many variants. The method is based on Generative Replay,
i.e. the use of generated samples to maintain past knowledge. It comes along
with a statistically sound method for environment change detection, which
self-triggers the Generative Replay. Our experiments on VAEs show that
S-TRIGGER learns state representations that allows fast and high-performing
Reinforcement Learning, while avoiding catastrophic forgetting. The resulting
system is capable of autonomously learning new information without using past
data and with a bounded system size. Code for our experiments is attached in
Appendix.
