Incremental Deep Neural Network Learning using Classification Confidence
  Thresholding
  Most modern neural networks for classification fail to take into account the
concept of the unknown. Trained neural networks are usually tested in an
unrealistic scenario with only examples from a closed set of known classes. In
an attempt to develop a more realistic model, the concept of working in an open
set environment has been introduced. This in turn leads to the concept of
incremental learning where a model with its own architecture and initial
trained set of data can identify unknown classes during the testing phase and
autonomously update itself if evidence of a new class is detected. Some
problems that arise in incremental learning are inefficient use of resources to
retrain the classifier repeatedly and the decrease of classification accuracy
as multiple classes are added over time. This process of instantiating new
classes is repeated as many times as necessary, accruing errors. To address
these problems, this paper proposes the Classification Confidence Threshold
approach to prime neural networks for incremental learning to keep accuracies
high by limiting forgetting. A lean method is also used to reduce resources
used in the retraining of the neural network. The proposed method is based on
the idea that a network is able to incrementally learn a new class even when
exposed to a limited number samples associated with the new class. This method
can be applied to most existing neural networks with minimal changes to network
architecture.
