A Hybrid GA-PSO Method for Evolving Architecture and Short Connections
  of Deep Convolutional Neural Networks
  Image classification is a difficult machine learning task, where
Convolutional Neural Networks (CNNs) have been applied for over 20 years in
order to solve the problem. In recent years, instead of the traditional way of
only connecting the current layer with its next layer, shortcut connections
have been proposed to connect the current layer with its forward layers apart
from its next layer, which has been proved to be able to facilitate the
training process of deep CNNs. However, there are various ways to build the
shortcut connections, it is hard to manually design the best shortcut
connections when solving a particular problem, especially given the design of
the network architecture is already very challenging.
  In this paper, a hybrid evolutionary computation (EC) method is proposed to
\textit{automatically} evolve both the architecture of deep CNNs and the
shortcut connections. Three major contributions of this work are: Firstly, a
new encoding strategy is proposed to encode a CNN, where the architecture and
the shortcut connections are encoded separately; Secondly, a hybrid two-level
EC method, which combines particle swarm optimisation and genetic algorithms,
is developed to search for the optimal CNNs; Lastly, an adjustable learning
rate is introduced for the fitness evaluations, which provides a better
learning rate for the training process given a fixed number of epochs. The
proposed algorithm is evaluated on three widely used benchmark datasets of
image classification and compared with 12 peer Non-EC based competitors and one
EC based competitor. The experimental results demonstrate that the proposed
method outperforms all of the peer competitors in terms of classification
accuracy.
