Lessons from the Clustering Analysis of a Search Space: A Centroid-based
  Approach to Initializing NAS
  Lots of effort in neural architecture search (NAS) research has been
dedicated to algorithmic development, aiming at designing more efficient and
less costly methods. Nonetheless, the investigation of the initialization of
these techniques remain scare, and currently most NAS methodologies rely on
stochastic initialization procedures, because acquiring information prior to
search is costly. However, the recent availability of NAS benchmarks have
enabled low computational resources prototyping. In this study, we propose to
accelerate a NAS algorithm using a data-driven initialization technique,
leveraging the availability of NAS benchmarks. Particularly, we proposed a
two-step methodology. First, a calibrated clustering analysis of the search
space is performed. Second, the centroids are extracted and used to initialize
a NAS algorithm. We tested our proposal using Aging Evolution, an evolutionary
algorithm, on NAS-bench-101. The results show that, compared to a random
initialization, a faster convergence and a better performance of the final
solution is achieved.
