Domino Saliency Metrics: Improving Existing Channel Saliency Metrics
  with Structural Information
  Channel pruning is used to reduce the number of weights in a Convolutional
Neural Network (CNN). Channel pruning removes slices of the weight tensor so
that the convolution layer remains dense. The removal of these weight slices
from a single layer causes mismatching number of feature maps between layers of
the network. A simple solution is to force the number of feature map between
layers to match through the removal of weight slices from subsequent layers.
This additional constraint becomes more apparent in DNNs with branches where
multiple channels need to be pruned together to keep the network dense. Popular
pruning saliency metrics do not factor in the structural dependencies that
arise in DNNs with branches. We propose Domino metrics (built on existing
channel saliency metrics) to reflect these structural constraints. We test
Domino saliency metrics against the baseline channel saliency metrics on
multiple networks with branches. Domino saliency metrics improved pruning rates
in most tested networks and up to 25% in AlexNet on CIFAR-10.
