Prune and Replace NAS
  While recent NAS algorithms are thousands of times faster than the pioneering
works, it is often overlooked that they use fewer candidate operations,
resulting in a significantly smaller search space. We present PR-DARTS, a NAS
algorithm that discovers strong network configurations in a much larger search
space and a single day. A small candidate operation pool is used, from which
candidates are progressively pruned and replaced with better performing ones.
Experiments on CIFAR-10 and CIFAR-100 achieve 2.51% and 15.53% test error,
respectively, despite searching in a space where each cell has 150 times as
many possible configurations than in the DARTS baseline. Code is available at
https://github.com/cogsys-tuebingen/prdarts
