Depthwise Separable Convolutions Allow for Fast and Memory-Efficient
  Spectral Normalization
  An increasing number of models require the control of the spectral norm of
convolutional layers of a neural network. While there is an abundance of
methods for estimating and enforcing upper bounds on those during training,
they are typically costly in either memory or time. In this work, we introduce
a very simple method for spectral normalization of depthwise separable
convolutions, which introduces negligible computational and memory overhead. We
demonstrate the effectiveness of our method on image classification tasks using
standard architectures like MobileNetV2.
