Neural Architecture Search via Bregman Iterations
  We propose a novel strategy for Neural Architecture Search (NAS) based on
Bregman iterations. Starting from a sparse neural network our gradient-based
one-shot algorithm gradually adds relevant parameters in an inverse scale space
manner. This allows the network to choose the best architecture in the search
space which makes it well-designed for a given task, e.g., by adding neurons or
skip connections. We demonstrate that using our approach one can unveil, for
instance, residual autoencoders for denoising, deblurring, and classification
tasks. Code is available at https://github.com/TimRoith/BregmanLearning.
