IC-Network: Efficient Structure for Convolutional Neural Networks
  Neural networks have been widely used, and most networks achieve excellent
performance by stacking certain types of basic units. Compared to increasing
the depth and width of the network, designing more effective basic units has
become an important research topic. Inspired by the elastic collision model in
physics, we present a universal structure that could be integrated into the
existing network structures to speed up the training process and increase their
generalization abilities. We term this structure the "Inter-layer Collision"
(IC) structure. We built two kinds of basic computational units (IC layer and
IC block) that compose the convolutional neural networks (CNNs) by combining
the IC structure with the convolution operation. Compared to traditional
convolutions, both of the proposed computational units have a stronger
non-linear representation ability and can filter features useful for a given
task. Using these computational units to build networks, we bring significant
improvements in performance for existing state-of-the-art CNNs. On the imagenet
experiment, we integrate the IC block into ResNet-50 and reduce the top-1 error
from 22.85% to 21.49%, which also exceeds the top-1 error of ResNet-100
(21.75%).
