Batch Normalization in Quantized Networks
  Implementation of quantized neural networks on computing hardware leads to
considerable speed up and memory saving. However, quantized deep networks are
difficult to train and batch~normalization (BatchNorm) layer plays an important
role in training full-precision and quantized networks. Most studies on
BatchNorm are focused on full-precision networks, and there is little research
in understanding BatchNorm affect in quantized training which we address here.
We show BatchNorm avoids gradient explosion which is counter-intuitive and
recently observed in numerical experiments by other researchers.
