S3NAS: Fast NPU-aware Neural Architecture Search Methodology
  As the application area of convolutional neural networks (CNN) is growing in
embedded devices, it becomes popular to use a hardware CNN accelerator, called
neural processing unit (NPU), to achieve higher performance per watt than CPUs
or GPUs. Recently, automated neural architecture search (NAS) emerges as the
default technique to find a state-of-the-art CNN architecture with higher
accuracy than manually-designed architectures for image classification. In this
paper, we present a fast NPU-aware NAS methodology, called S3NAS, to find a CNN
architecture with higher accuracy than the existing ones under a given latency
constraint. It consists of three steps: supernet design, Single-Path NAS for
fast architecture exploration, and scaling. To widen the search space of the
supernet structure that consists of stages, we allow stages to have a different
number of blocks and blocks to have parallel layers of different kernel sizes.
For a fast neural architecture search, we apply a modified Single-Path NAS
technique to the proposed supernet structure. In this step, we assume a shorter
latency constraint than the required to reduce the search space and the search
time. The last step is to scale up the network maximally within the latency
constraint. For accurate latency estimation, an analytical latency estimator is
devised, based on a cycle-level NPU simulator that runs an entire CNN
considering the memory access overhead accurately. With the proposed
methodology, we are able to find a network in 3 hours using TPUv3, which shows
82.72% top-1 accuracy on ImageNet with 11.66 ms latency. Code are released at
https://github.com/cap-lab/S3NAS
