Online Learned Continual Compression with Adaptive Quantization Modules
  We introduce and study the problem of Online Continual Compression, where one
attempts to simultaneously learn to compress and store a representative dataset
from a non i.i.d data stream, while only observing each sample once. A naive
application of auto-encoders in this setting encounters a major challenge:
representations derived from earlier encoder states must be usable by later
decoder states. We show how to use discrete auto-encoders to effectively
address this challenge and introduce Adaptive Quantization Modules (AQM) to
control variation in the compression ability of the module at any given stage
of learning. This enables selecting an appropriate compression for incoming
samples, while taking into account overall memory constraints and current
progress of the learned compression. Unlike previous methods, our approach does
not require any pretraining, even on challenging datasets. We show that using
AQM to replace standard episodic memory in continual learning settings leads to
significant gains on continual learning benchmarks. Furthermore we demonstrate
this approach with larger images, LiDAR, and reinforcement learning
environments.
