Subset Sampling For Progressive Neural Network Learning
  Progressive Neural Network Learning is a class of algorithms that
incrementally construct the network's topology and optimize its parameters
based on the training data. While this approach exempts the users from the
manual task of designing and validating multiple network topologies, it often
requires an enormous number of computations. In this paper, we propose to speed
up this process by exploiting subsets of training data at each incremental
training step. Three different sampling strategies for selecting the training
samples according to different criteria are proposed and evaluated. We also
propose to perform online hyperparameter selection during the network
progression, which further reduces the overall training time. Experimental
results in object, scene and face recognition problems demonstrate that the
proposed approach speeds up the optimization procedure considerably while
operating on par with the baseline approach exploiting the entire training set
throughout the training process.
