Trainable Spectrally Initializable Matrix Transformations in
  Convolutional Neural Networks
  In this work, we investigate the application of trainable and spectrally
initializable matrix transformations on the feature maps produced by
convolution operations. While previous literature has already demonstrated the
possibility of adding static spectral transformations as feature processors,
our focus is on more general trainable transforms. We study the transforms in
various architectural configurations on four datasets of different nature: from
medical (ColorectalHist, HAM10000) and natural (Flowers, ImageNet) images to
historical documents (CB55) and handwriting recognition (GPDS). With rigorous
experiments that control for the number of parameters and randomness, we show
that networks utilizing the introduced matrix transformations outperform
vanilla neural networks. The observed accuracy increases by an average of 2.2
across all datasets. In addition, we show that the benefit of spectral
initialization leads to significantly faster convergence, as opposed to
randomly initialized matrix transformations. The transformations are
implemented as auto-differentiable PyTorch modules that can be incorporated
into any neural network architecture. The entire code base is open-source.
