FQ-Conv: Fully Quantized Convolution for Efficient and Accurate
  Inference
  Deep neural networks (DNNs) can be made hardware-efficient by reducing the
numerical precision of the weights and activations of the network and by
improving the network's resilience to noise. However, this gain in efficiency
often comes at the cost of significantly reduced accuracy. In this paper, we
present a novel approach to quantizing convolutional neural network. The
resulting networks perform all computations in low-precision, without requiring
higher-precision BN and nonlinearities, while still being highly accurate. To
achieve this result, we employ a novel quantization technique that learns to
optimally quantize the weights and activations of the network during training.
Additionally, to enhance training convergence we use a new training technique,
called gradual quantization. We leverage the nonlinear and normalizing behavior
of our quantization function to effectively remove the higher-precision
nonlinearities and BN from the network. The resulting convolutional layers are
fully quantized to low precision, from input to output, ideal for neural
network accelerators on the edge. We demonstrate the potential of this approach
on different datasets and networks, showing that ternary-weight CNNs with
low-precision in- and outputs perform virtually on par with their
full-precision equivalents. Finally, we analyze the influence of noise on the
weights, activations and convolution outputs (multiply-accumulate, MAC) and
propose a strategy to improve network performance under noisy conditions.
