Unified Probabilistic Deep Continual Learning through Generative Replay
  and Open Set Recognition
  Modern deep neural networks are well known to be brittle in the face of
unknown data instances and recognition of the latter remains a challenge.
Although it is inevitable for continual-learning systems to encounter such
unseen concepts, the corresponding literature appears to nonetheless focus
primarily on alleviating catastrophic interference with learned
representations. In this work, we introduce a probabilistic approach that
connects these perspectives based on variational inference in a single deep
autoencoder model. Specifically, we propose to bound the approximate posterior
by fitting regions of high density on the basis of correctly classified data
points. These bounds are shown to serve a dual purpose: unseen unknown
out-of-distribution data can be distinguished from already trained known tasks
towards robust application. Simultaneously, to retain already acquired
knowledge, a generative replay process can be narrowed to strictly
in-distribution samples, in order to significantly alleviate catastrophic
interference.
