Metaheuristic Design of Feedforward Neural Networks: A Review of Two
  Decades of Research
  Over the past two decades, the feedforward neural network (FNN) optimization
has been a key interest among the researchers and practitioners of multiple
disciplines. The FNN optimization is often viewed from the various
perspectives: the optimization of weights, network architecture, activation
nodes, learning parameters, learning environment, etc. Researchers adopted such
different viewpoints mainly to improve the FNN's generalization ability. The
gradient-descent algorithm such as backpropagation has been widely applied to
optimize the FNNs. Its success is evident from the FNN's application to
numerous real-world problems. However, due to the limitations of the
gradient-based optimization methods, the metaheuristic algorithms including the
evolutionary algorithms, swarm intelligence, etc., are still being widely
explored by the researchers aiming to obtain generalized FNN for a given
problem. This article attempts to summarize a broad spectrum of FNN
optimization methodologies including conventional and metaheuristic approaches.
This article also tries to connect various research directions emerged out of
the FNN optimization practices, such as evolving neural network (NN),
cooperative coevolution NN, complex-valued NN, deep learning, extreme learning
machine, quantum NN, etc. Additionally, it provides interesting research
challenges for future research to cope-up with the present information
processing era.
