Channel Compression: Rethinking Information Redundancy among Channels in
  CNN Architecture
  Model compression and acceleration are attracting increasing attentions due
to the demand for embedded devices and mobile applications. Research on
efficient convolutional neural networks (CNNs) aims at removing feature
redundancy by decomposing or optimizing the convolutional calculation. In this
work, feature redundancy is assumed to exist among channels in CNN
architectures, which provides some leeway to boost calculation efficiency.
Aiming at channel compression, a novel convolutional construction named compact
convolution is proposed to embrace the progress in spatial convolution, channel
grouping and pooling operation. Specifically, the depth-wise separable
convolution and the point-wise interchannel operation are utilized to
efficiently extract features. Different from the existing channel compression
method which usually introduces considerable learnable weights, the proposed
compact convolution can reduce feature redundancy with no extra parameters.
With the point-wise interchannel operation, compact convolutions implicitly
squeeze the channel dimension of feature maps. To explore the rules on reducing
channel redundancy in neural networks, the comparison is made among different
point-wise interchannel operations. Moreover, compact convolutions are extended
to tackle with multiple tasks, such as acoustic scene classification, sound
event detection and image classification. The extensive experiments demonstrate
that our compact convolution not only exhibits high effectiveness in several
multimedia tasks, but also can be efficiently implemented by benefiting from
parallel computation.
