Compressive sensing with un-trained neural networks: Gradient descent
  finds the smoothest approximation
  Un-trained convolutional neural networks have emerged as highly successful
tools for image recovery and restoration. They are capable of solving standard
inverse problems such as denoising and compressive sensing with excellent
results by simply fitting a neural network model to measurements from a single
image or signal without the need for any additional training data. For some
applications, this critically requires additional regularization in the form of
early stopping the optimization. For signal recovery from a few measurements,
however, un-trained convolutional networks have an intriguing self-regularizing
property: Even though the network can perfectly fit any image, the network
recovers a natural image from few measurements when trained with gradient
descent until convergence. In this paper, we provide numerical evidence for
this property and study it theoretically. We show that---without any further
regularization---an un-trained convolutional neural network can approximately
reconstruct signals and images that are sufficiently structured, from a near
minimal number of random measurements.
