Jointly Sparse Convolutional Neural Networks in Dual Spatial-Winograd
  Domains
  We consider the optimization of deep convolutional neural networks (CNNs)
such that they provide good performance while having reduced complexity if
deployed on either conventional systems with spatial-domain convolution or
lower-complexity systems designed for Winograd convolution. The proposed
framework produces one compressed model whose convolutional filters can be made
sparse either in the spatial domain or in the Winograd domain. Hence, the
compressed model can be deployed universally on any platform, without need for
re-training on the deployed platform. To get a better compression ratio, the
sparse model is compressed in the spatial domain that has a fewer number of
parameters. From our experiments, we obtain $24.2\times$ and $47.7\times$
compressed models for ResNet-18 and AlexNet trained on the ImageNet dataset,
while their computational cost is also reduced by $4.5\times$ and $5.1\times$,
respectively.
