ImmuNeCS: Neural Committee Search by an Artificial Immune System
  Current Neural Architecture Search techniques can suffer from a few
shortcomings, including high computational cost, excessive bias from the search
space, conceptual complexity or uncertain empirical benefits over random
search. In this paper, we present ImmuNeCS, an attempt at addressing these
issues with a method that offers a simple, flexible, and efficient way of
building deep learning models automatically, and we demonstrate its
effectiveness in the context of convolutional neural networks. Instead of
searching for the 1-best architecture for a given task, we focus on building a
population of neural networks that are then ensembled into a neural network
committee, an approach we dub 'Neural Committee Search'. To ensure sufficient
performance from the committee, our search algorithm is based on an artificial
immune system that balances individual performance with population diversity.
This allows us to stop the search when accuracy starts to plateau, and to
bridge the performance gap through ensembling. In order to justify our method,
we first verify that the chosen search space exhibits the locality property. To
further improve efficiency, we also combine partial evaluation, weight
inheritance, and progressive search. First, experiments are run to verify the
validity of these techniques. Then, preliminary experimental results on two
popular computer vision benchmarks show that our method consistently
outperforms random search and yields promising results within reasonable GPU
budgets. An additional experiment also shows that ImmuNeCS's solutions transfer
effectively to a more difficult task, where they achieve results comparable to
a direct search on the new task. We believe these findings can open the way for
new, accessible alternatives to traditional NAS.
