D-DARTS: Distributed Differentiable Architecture Search
  Differentiable ARchiTecture Search (DARTS) is one of the most trending Neural
Architecture Search (NAS) methods, drastically reducing search cost by
resorting to Stochastic Gradient Descent (SGD) and weight-sharing. However, it
also greatly reduces the search space, thus excluding potential promising
architectures from being discovered. In this paper, we propose D-DARTS, a novel
solution that addresses this problem by nesting several neural networks at
cell-level instead of using weight-sharing to produce more diversified and
specialized architectures. Moreover, we introduce a novel algorithm which can
derive deeper architectures from a few trained cells, increasing performance
and saving computation time. Our solution is able to provide state-of-the-art
results on CIFAR-10, CIFAR-100 and ImageNet while using significantly less
parameters than previous baselines, resulting in more hardware-efficient neural
networks.
