Convolutional Neural Networks using Logarithmic Data Representation
  Recent advances in convolutional neural networks have considered model
complexity and hardware efficiency to enable deployment onto embedded systems
and mobile devices. For example, it is now well-known that the arithmetic
operations of deep networks can be encoded down to 8-bit fixed-point without
significant deterioration in performance. However, further reduction in
precision down to as low as 3-bit fixed-point results in significant losses in
performance. In this paper we propose a new data representation that enables
state-of-the-art networks to be encoded to 3 bits with negligible loss in
classification performance. To perform this, we take advantage of the fact that
the weights and activations in a trained network naturally have non-uniform
distributions. Using non-uniform, base-2 logarithmic representation to encode
weights, communicate activations, and perform dot-products enables networks to
1) achieve higher classification accuracies than fixed-point at the same
resolution and 2) eliminate bulky digital multipliers. Finally, we propose an
end-to-end training procedure that uses log representation at 5-bits, which
achieves higher final test accuracy than linear at 5-bits.
