Translating Diffusion, Wavelets, and Regularisation into Residual
  Networks
  Convolutional neural networks (CNNs) often perform well, but their stability
is poorly understood. To address this problem, we consider the simple
prototypical problem of signal denoising, where classical approaches such as
nonlinear diffusion, wavelet-based methods and regularisation offer provable
stability guarantees. To transfer such guarantees to CNNs, we interpret
numerical approximations of these classical methods as a specific residual
network (ResNet) architecture. This leads to a dictionary which allows to
translate diffusivities, shrinkage functions, and regularisers into activation
functions, and enables a direct communication between the four research
communities. On the CNN side, it does not only inspire new families of
nonmonotone activation functions, but also introduces intrinsically stable
architectures for an arbitrary number of layers.
