Going Beyond Neural Architecture Search with Sampling-based Neural
  Ensemble Search
  Recently, Neural Architecture Search (NAS) has been widely applied to
automate the design of deep neural networks. Various NAS algorithms have been
proposed to reduce the search cost and improve the generalization performance
of those final selected architectures. However, these NAS algorithms aim to
select only a single neural architecture from the search spaces and thus have
overlooked the capability of other candidate architectures in helping improve
the performance of their final selected architecture. To this end, we present
two novel sampling algorithms under our Neural Ensemble Search via Sampling
(NESS) framework that can effectively and efficiently select a well-performing
ensemble of neural architectures from NAS search space. Compared with
state-of-the-art NAS algorithms and other well-known ensemble search baselines,
our NESS algorithms are shown to be able to achieve improved performance in
both classification and adversarial defense tasks on various benchmark datasets
while incurring a comparable search cost to these NAS algorithms.
