On regularization for a convolutional kernel in neural networks
  Convolutional neural network is an important model in deep learning. To avoid
exploding/vanishing gradient problems and to improve the generalizability of a
neural network, it is desirable to have a convolution operation that nearly
preserves the norm, or to have the singular values of the transformation matrix
corresponding to a convolutional kernel bounded around $1$. We propose a
penalty function that can be used in the optimization of a convolutional neural
network to constrain the singular values of the transformation matrix around
$1$. We derive an algorithm to carry out the gradient descent minimization of
this penalty function in terms of convolution kernels. Numerical examples are
presented to demonstrate the effectiveness of the method.
