Sequential training algorithm for neural networks
  A sequential training method for large-scale feedforward neural networks is
presented. Each layer of the neural network is decoupled and trained
separately. After the training is completed for each layer, they are combined
together. The performance of the network would be sub-optimal compared to the
full network training if the optimal solution would be achieved. However,
achieving the optimal solution for the full network would be infeasible or
require long computing time. The proposed sequential approach reduces the
required computer resources significantly and would have better convergences as
a single layer is optimised for each optimisation step. The required
modifications of existing algorithms to implement the sequential training are
minimal. The performance is verified by a simple example.
