The Singular Values of Convolutional Layers
  We characterize the singular values of the linear transformation associated
with a standard 2D multi-channel convolutional layer, enabling their efficient
computation. This characterization also leads to an algorithm for projecting a
convolutional layer onto an operator-norm ball. We show that this is an
effective regularizer; for example, it improves the test error of a deep
residual network using batch normalization on CIFAR-10 from 6.2\% to 5.3\%.
