Incremental Class Learning using Variational Autoencoders with
  Similarity Learning
  Catastrophic forgetting in neural networks during incremental learning
remains a challenging problem. Previous research investigated catastrophic
forgetting in fully connected networks, with some earlier work exploring
activation functions and learning algorithms. Applications of neural networks
have been extended to include similarity learning. It is of significant
interest to understand how similarity learning loss functions would be affected
by catastrophic forgetting. Our research investigates catastrophic forgetting
for four well-known similarity-based loss functions during incremental class
learning. The loss functions are angular, contrastive, centre, and triplet
loss. Our results show that the rate of catastrophic forgetting is different
across loss functions on multiple datasets. The angular loss was least
affected, followed by contrastive, triplet loss, and centre loss with good
mining techniques. We implemented three existing incremental learning
techniques, iCaRL, EWC, and EBLL. We further proposed our novel technique using
VAEs to generate representation as exemplars that are passed through
intermediate layers of the network. Our method outperformed the three existing
techniques. We have shown that we do not require stored images as exemplars for
incremental learning with similarity learning. The generated representations
can help preserve regions of the embedding space used by prior knowledge so
that new knowledge will not ``overwrite'' prior knowledge.
