DARTS for Inverse Problems: a Study on Hyperparameter Sensitivity
  Differentiable architecture search (DARTS) is a widely researched tool for
the discovery of novel architectures, due to its promising results for image
classification. The main benefit of DARTS is the effectiveness achieved through
the weight-sharing one-shot paradigm, which allows efficient architecture
search. In this work, we investigate DARTS in a systematic case study of
inverse problems, which allows us to analyze these potential benefits in a
controlled manner. We demonstrate that the success of DARTS can be extended
from image classification to signal reconstruction, in principle. However, our
experiments also expose three fundamental difficulties in the evaluation of
DARTS-based methods in inverse problems: First, the results show a large
variance in all test cases. Second, the final performance is highly dependent
on the hyperparameters of the optimizer. And third, the performance of the
weight-sharing architecture used during training does not reflect the final
performance of the found architecture well. Thus, we conclude the necessity to
1) report the results of any DARTS-based methods from several runs along with
its underlying performance statistics, 2) show the correlation of the training
and final architecture performance, and 3) carefully consider if the
computational efficiency of DARTS outweighs the costs of hyperparameter
optimization and multiple runs.
