ProxyBO: Accelerating Neural Architecture Search via Bayesian
  Optimization with Zero-cost Proxies
  Designing neural architectures requires immense manual efforts. This has
promoted the development of neural architecture search (NAS) to automate this
design. While previous NAS methods achieve promising results but run slowly and
zero-cost proxies run extremely fast but are less promising, recent work
considers utilizing zero-cost proxies via a simple warm-up. The existing method
has two limitations, which are unforeseeable reliability and one-shot usage. To
address the limitations, we present ProxyBO, an efficient Bayesian optimization
framework that utilizes the zero-cost proxies to accelerate neural architecture
search. We propose the generalization ability measurement to estimate the
fitness of proxies on the task during each iteration and then combine BO with
zero-cost proxies via dynamic influence combination. Extensive empirical
studies show that ProxyBO consistently outperforms competitive baselines on
five tasks from three public benchmarks. Concretely, ProxyBO achieves up to
5.41x and 3.83x speedups over the state-of-the-art approach REA and BRP-NAS,
respectively.
