Multi-objective Evolutionary Approach for Efficient Kernel Size and
  Shape for CNN
  While state-of-the-art development in CNN topology, such as VGGNet and
ResNet, have become increasingly accurate, these networks are computationally
expensive involving billions of arithmetic operations and parameters. To
improve the classification accuracy, state-of-the-art CNNs usually involve
large and complex convolutional layers. However, for certain applications, e.g.
Internet of Things (IoT), where such CNNs are to be implemented on
resource-constrained platforms, the CNN architectures have to be small and
efficient. To deal with this problem, reducing the resource consumption in
convolutional layers has become one of the most significant solutions. In this
work, a multi-objective optimisation approach is proposed to trade-off between
the amount of computation and network accuracy by using Multi-Objective
Evolutionary Algorithms (MOEAs). The number of convolution kernels and the size
of these kernels are proportional to computational resource consumption of
CNNs. Therefore, this paper considers optimising the computational resource
consumption by reducing the size and number of kernels in convolutional layers.
Additionally, the use of unconventional kernel shapes has been investigated and
results show these clearly outperform the commonly used square convolution
kernels. The main contributions of this paper are therefore a methodology to
significantly reduce computational cost of CNNs, based on unconventional kernel
shapes, and provide different trade-offs for specific use cases. The
experimental results further demonstrate that the proposed method achieves
large improvements in resource consumption with no significant reduction in
network performance. Compared with the benchmark CNN, the best trade-off
architecture shows a reduction in multiplications of up to 6X and with slight
increase in classification accuracy on CIFAR-10 dataset.
