Dynamic Clone Transformer for Efficient Convolutional Neural Netwoks
  Convolutional networks (ConvNets) have shown impressive capability to solve
various vision tasks. Nevertheless, the trade-off between performance and
efficiency is still a challenge for a feasible model deployment on
resource-constrained platforms. In this paper, we introduce a novel concept
termed multi-path fully connected pattern (MPFC) to rethink the
interdependencies of topology pattern, accuracy and efficiency for ConvNets.
Inspired by MPFC, we further propose a dual-branch module named dynamic clone
transformer (DCT) where one branch generates multiple replicas from inputs and
another branch reforms those clones through a series of difference vectors
conditional on inputs itself to produce more variants. This operation allows
the self-expansion of channel-wise information in a data-driven way with little
computational cost while providing sufficient learning capacity, which is a
potential unit to replace computationally expensive pointwise convolution as an
expansion layer in the bottleneck structure.
