A Comprehensive Survey on Hardware-Aware Neural Architecture Search
  Neural Architecture Search (NAS) methods have been growing in popularity.
These techniques have been fundamental to automate and speed up the time
consuming and error-prone process of synthesizing novel Deep Learning (DL)
architectures. NAS has been extensively studied in the past few years. Arguably
their most significant impact has been in image classification and object
detection tasks where the state of the art results have been obtained. Despite
the significant success achieved to date, applying NAS to real-world problems
still poses significant challenges and is not widely practical. In general, the
synthesized Convolution Neural Network (CNN) architectures are too complex to
be deployed in resource-limited platforms, such as IoT, mobile, and embedded
systems. One solution growing in popularity is to use multi-objective
optimization algorithms in the NAS search strategy by taking into account
execution latency, energy consumption, memory footprint, etc. This kind of NAS,
called hardware-aware NAS (HW-NAS), makes searching the most efficient
architecture more complicated and opens several questions.
  In this survey, we provide a detailed review of existing HW-NAS research and
categorize them according to four key dimensions: the search space, the search
strategy, the acceleration technique, and the hardware cost estimation
strategies. We further discuss the challenges and limitations of existing
approaches and potential future directions. This is the first survey paper
focusing on hardware-aware NAS. We hope it serves as a valuable reference for
the various techniques and algorithms discussed and paves the road for future
research towards hardware-aware NAS.
