When deep learning models on GPU can be accelerated by taking advantage
  of unstructured sparsity
  This paper is focused on the improvement the efficiency of the sparse
convolutional neural networks (CNNs) layers on graphic processing units (GPU).
The Nvidia deep neural network (cuDnn) library provides the most effective
implementation of deep learning (DL) algorithms for GPUs. GPUs are one of the
most efficient and commonly used accelerators for deep learning computations.
The modern CNN models need megabytes of coefficients and needed millions MAC
operations to perform convolution. One of the most common techniques for
compressing CNN models is weight pruning. There are two main types of pruning:
structural (based on removing whole weight channels) and non-structural
(removing individual weights). The first enables much easier acceleration, but
with this type it is difficult to achieve a sparsity level and accuracy as high
as that obtained with the second type. Non-structural pruning with retraining
can generate a matrix-weight up to $\sim90\%$ or more of sparsity in some deep
CNN models. This work shows when is worth using a direct sparse operation to
speed-up the calculation of the convolution layers. The VGG-16, CNN-non-static
and 1x1 layers from ResNet models were used as a benchmarks. In addition, we
present the impact of using reduced precision on time efficiency.
