Understanding Geometry of Encoder-Decoder CNNs
  Encoder-decoder networks using convolutional neural network (CNN)
architecture have been extensively used in deep learning literatures thanks to
its excellent performance for various inverse problems. However, it is still
difficult to obtain coherent geometric view why such an architecture gives the
desired performance. Inspired by recent theoretical understanding on
generalizability, expressivity and optimization landscape of neural networks,
as well as the theory of convolutional framelets, here we provide a unified
theoretical framework that leads to a better understanding of geometry of
encoder-decoder CNNs. Our unified mathematical framework shows that
encoder-decoder CNN architecture is closely related to nonlinear basis
representation using combinatorial convolution frames, whose expressibility
increases exponentially with the network depth. We also demonstrate the
importance of skipped connection in terms of expressibility, and optimization
landscape.
