Continual learning using hash-routed convolutional neural networks
  Continual learning could shift the machine learning paradigm from data
centric to model centric. A continual learning model needs to scale efficiently
to handle semantically different datasets, while avoiding unnecessary growth.
We introduce hash-routed convolutional neural networks: a group of
convolutional units where data flows dynamically. Feature maps are compared
using feature hashing and similar data is routed to the same units. A
hash-routed network provides excellent plasticity thanks to its routed nature,
while generating stable features through the use of orthogonal feature hashing.
Each unit evolves separately and new units can be added (to be used only when
necessary). Hash-routed networks achieve excellent performance across a variety
of typical continual learning benchmarks without storing raw data and train
using only gradient descent. Besides providing a continual learning framework
for supervised tasks with encouraging results, our model can be used for
unsupervised or reinforcement learning.
