Multi-scale Convolution Aggregation and Stochastic Feature Reuse for
  DenseNets
  Recently, Convolution Neural Networks (CNNs) obtained huge success in
numerous vision tasks. In particular, DenseNets have demonstrated that feature
reuse via dense skip connections can effectively alleviate the difficulty of
training very deep networks and that reusing features generated by the initial
layers in all subsequent layers has strong impact on performance. To feed even
richer information into the network, a novel adaptive Multi-scale Convolution
Aggregation module is presented in this paper. Composed of layers for
multi-scale convolutions, trainable cross-scale aggregation, maxout, and
concatenation, this module is highly non-linear and can boost the accuracy of
DenseNet while using much fewer parameters. In addition, due to high model
complexity, the network with extremely dense feature reuse is prone to
overfitting. To address this problem, a regularization method named Stochastic
Feature Reuse is also presented. Through randomly dropping a set of feature
maps to be reused for each mini-batch during the training phase, this
regularization method reduces training costs and prevents co-adaptation.
Experimental results on CIFAR-10, CIFAR-100 and SVHN benchmarks demonstrated
the effectiveness of the proposed methods.
