DSXplore: Optimizing Convolutional Neural Networks via Sliding-Channel
  Convolutions
  As the key advancement of the convolutional neural networks (CNNs), depthwise
separable convolutions (DSCs) are becoming one of the most popular techniques
to reduce the computations and parameters size of CNNs meanwhile maintaining
the model accuracy. It also brings profound impact to improve the applicability
of the compute- and memory-intensive CNNs to a broad range of applications,
such as mobile devices, which are generally short of computation power and
memory. However, previous research in DSCs are largely focusing on compositing
the limited existing DSC designs, thus, missing the opportunities to explore
more potential designs that can achieve better accuracy and higher
computation/parameter reduction. Besides, the off-the-shelf convolution
implementations offer limited computing schemes, therefore, lacking support for
DSCs with different convolution patterns.
  To this end, we introduce, DSXplore, the first optimized design for exploring
DSCs on CNNs. Specifically, at the algorithm level, DSXplore incorporates a
novel factorized kernel -- sliding-channel convolution (SCC), featured with
input-channel overlapping to balance the accuracy performance and the reduction
of computation and memory cost. SCC also offers enormous space for design
exploration by introducing adjustable kernel parameters. Further, at the
implementation level, we carry out an optimized GPU-implementation tailored for
SCC by leveraging several key techniques, such as the input-centric backward
design and the channel-cyclic optimization. Intensive experiments on different
datasets across mainstream CNNs show the advantages of DSXplore in balancing
accuracy and computation/parameter reduction over the standard convolution and
the existing DSCs.
