Optimal Quantization for Batch Normalization in Neural Network
  Deployments and Beyond
  Quantized Neural Networks (QNNs) use low bit-width fixed-point numbers for
representing weight parameters and activations, and are often used in
real-world applications due to their saving of computation resources and
reproducibility of results.
  Batch Normalization (BN) poses a challenge for QNNs for requiring floating
points in reciprocal operations, and previous QNNs either require computing BN
at high precision or revise BN to some variants in heuristic ways.
  In this work, we propose a novel method to quantize BN by converting an
affine transformation of two floating points to a fixed-point operation with
shared quantized scale, which is friendly for hardware acceleration and model
deployment.
  We confirm that our method maintains same outputs through rigorous
theoretical analysis and numerical analysis. Accuracy and efficiency of our
quantization method are verified by experiments at layer level on CIFAR and
ImageNet datasets.
  We also believe that our method is potentially useful in other problems
involving quantization.
