A Comprehensive Survey of Neural Architecture Search: Challenges and
  Solutions
  Deep learning has made breakthroughs and substantial in many fields due to
its powerful automatic representation capabilities. It has been proven that
neural architecture design is crucial to the feature representation of data and
the final performance. However, the design of the neural architecture heavily
relies on the researchers' prior knowledge and experience. And due to the
limitations of human' inherent knowledge, it is difficult for people to jump
out of their original thinking paradigm and design an optimal model. Therefore,
an intuitive idea would be to reduce human intervention as much as possible and
let the algorithm automatically design the neural architecture. Neural
Architecture Search (NAS) is just such a revolutionary algorithm, and the
related research work is complicated and rich. Therefore, a comprehensive and
systematic survey on the NAS is essential. Previously related surveys have
begun to classify existing work mainly based on the key components of NAS:
search space, search strategy, and evaluation strategy. While this
classification method is more intuitive, it is difficult for readers to grasp
the challenges and the landmark work involved. Therefore, in this survey, we
provide a new perspective: beginning with an overview of the characteristics of
the earliest NAS algorithms, summarizing the problems in these early NAS
algorithms, and then providing solutions for subsequent related research work.
Besides, we conduct a detailed and comprehensive analysis, comparison, and
summary of these works. Finally, we provide some possible future research
directions.
