Convolutional Neural Networks Analyzed via Inverse Problem Theory and
  Sparse Representations
  Inverse problems in imaging such as denoising, deblurring, superresolution
(SR) have been addressed for many decades. In recent years, convolutional
neural networks (CNNs) have been widely used for many inverse problem areas.
Although their indisputable success, CNNs are not mathematically validated as
to how and what they learn. In this paper, we prove that during training, CNN
elements solve for inverse problems which are optimum solutions stored as CNN
neuron filters. We discuss the necessity of mutual coherence between CNN layer
elements in order for a network to converge to the optimum solution. We prove
that required mutual coherence can be provided by the usage of residual
learning and skip connections. We have set rules over training sets and depth
of networks for better convergence, i.e. performance.
