ACDC: Weight Sharing in Atom-Coefficient Decomposed Convolution
  Convolutional Neural Networks (CNNs) are known to be significantly
over-parametrized, and difficult to interpret, train and adapt. In this paper,
we introduce a structural regularization across convolutional kernels in a CNN.
In our approach, each convolution kernel is first decomposed as 2D dictionary
atoms linearly combined by coefficients. The widely observed correlation and
redundancy in a CNN hint a common low-rank structure among the decomposed
coefficients, which is here further supported by our empirical observations. We
then explicitly regularize CNN kernels by enforcing decomposed coefficients to
be shared across sub-structures, while leaving each sub-structure only its own
dictionary atoms, a few hundreds of parameters typically, which leads to
dramatic model reductions. We explore models with sharing across different
sub-structures to cover a wide range of trade-offs between parameter reduction
and expressiveness. Our proposed regularized network structures open the door
to better interpreting, training and adapting deep models. We validate the
flexibility and compatibility of our method by image classification experiments
on multiple datasets and underlying network structures, and show that CNNs now
maintain performance with dramatic reduction in parameters and computations,
e.g., only 5\% parameters are used in a ResNet-18 to achieve comparable
performance. Further experiments on few-shot classification show that faster
and more robust task adaptation is obtained in comparison with models with
standard convolutions.
