Optimizing Deep Neural Network Architecture: A Tabu Search Based
  Approach
  The performance of Feedforward neural network (FNN) fully de-pends upon the
selection of architecture and training algorithm. FNN architecture can be
tweaked using several parameters, such as the number of hidden layers, number
of hidden neurons at each hidden layer and number of connections between
layers. There may be exponential combinations for these architectural
attributes which may be unmanageable manually, so it requires an algorithm
which can automatically design an optimal architecture with high generalization
ability. Numerous optimization algorithms have been utilized for FNN
architecture determination. This paper proposes a new methodology which can
work on the estimation of hidden layers and their respective neurons for FNN.
This work combines the advantages of Tabu search (TS) and Gradient descent with
momentum backpropagation (GDM) training algorithm to demonstrate how Tabu
search can automatically select the best architecture from the populated
architectures based on minimum testing error criteria. The proposed approach
has been tested on four classification benchmark dataset of different size.
