Finet: Using Fine-grained Batch Normalization to Train Light-weight
  Neural Networks
  To build light-weight network, we propose a new normalization, Fine-grained
Batch Normalization (FBN). Different from Batch Normalization (BN), which
normalizes the final summation of the weighted inputs, FBN normalizes the
intermediate state of the summation. We propose a novel light-weight network
based on FBN, called Finet. At training time, the convolutional layer with FBN
can be seen as an inverted bottleneck mechanism. FBN can be fused into
convolution at inference time. After fusion, Finet uses the standard
convolution with equal channel width, thus makes the inference more efficient.
On ImageNet classification dataset, Finet achieves the state-of-art performance
(65.706% accuracy with 43M FLOPs, and 73.786% accuracy with 303M FLOPs),
Moreover, experiments show that Finet is more efficient than other state-of-art
light-weight networks.
