Surrogate-assisted Particle Swarm Optimisation for Evolving
  Variable-length Transferable Blocks for Image Classification
  Deep convolutional neural networks have demonstrated promising performance on
image classification tasks, but the manual design process becomes more and more
complex due to the fast depth growth and the increasingly complex topologies of
convolutional neural networks. As a result, neural architecture search has
emerged to automatically design convolutional neural networks that outperform
handcrafted counterparts. However, the computational cost is immense, e.g.
22,400 GPU-days and 2,000 GPU-days for two outstanding neural architecture
search works named NAS and NASNet, respectively, which motivates this work. A
new effective and efficient surrogate-assisted particle swarm optimisation
algorithm is proposed to automatically evolve convolutional neural networks.
This is achieved by proposing a novel surrogate model, a new method of creating
a surrogate dataset and a new encoding strategy to encode variable-length
blocks of convolutional neural networks, all of which are integrated into a
particle swarm optimisation algorithm to form the proposed method. The proposed
method shows its effectiveness by achieving competitive error rates of 3.49% on
the CIFAR-10 dataset, 18.49% on the CIFAR-100 dataset, and 1.82% on the SVHN
dataset. The convolutional neural network blocks are efficiently learned by the
proposed method from CIFAR-10 within 3 GPU-days due to the acceleration
achieved by the surrogate model and the surrogate dataset to avoid the training
of 80.1% of convolutional neural network blocks represented by the particles.
Without any further search, the evolved blocks from CIFAR-10 can be
successfully transferred to CIFAR-100 and SVHN, which exhibits the
transferability of the block learned by the proposed method.
