Compressive Sensing and Neural Networks from a Statistical Learning
  Perspective
  Various iterative reconstruction algorithms for inverse problems can be
unfolded as neural networks. Empirically, this approach has often led to
improved results, but theoretical guarantees are still scarce. While some
progress on generalization properties of neural networks have been made, great
challenges remain. In this chapter, we discuss and combine these topics to
present a generalization error analysis for a class of neural networks suitable
for sparse reconstruction from few linear measurements. The hypothesis class
considered is inspired by the classical iterative soft-thresholding algorithm
(ISTA). The neural networks in this class are obtained by unfolding iterations
of ISTA and learning some of the weights. Based on training samples, we aim at
learning the optimal network parameters via empirical risk minimization and
thereby the optimal network that reconstructs signals from their compressive
linear measurements. In particular, we may learn a sparsity basis that is
shared by all of the iterations/layers and thereby obtain a new approach for
dictionary learning. For this class of networks, we present a generalization
bound, which is based on bounding the Rademacher complexity of hypothesis
classes consisting of such deep networks via Dudley's integral. Remarkably,
under realistic conditions, the generalization error scales only
logarithmically in the number of layers, and at most linear in number of
measurements.
