Bit-pragmatic Deep Neural Network Computing
  We quantify a source of ineffectual computations when processing the
multiplications of the convolutional layers in Deep Neural Networks (DNNs) and
propose Pragmatic (PRA), an architecture that exploits it improving performance
and energy efficiency. The source of these ineffectual computations is best
understood in the context of conventional multipliers which generate internally
multiple terms, that is, products of the multiplicand and powers of two, which
added together produce the final product [1]. At runtime, many of these terms
are zero as they are generated when the multiplicand is combined with the
zero-bits of the multiplicator. While conventional bit-parallel multipliers
calculate all terms in parallel to reduce individual product latency, PRA
calculates only the non-zero terms using a) on-the-fly conversion of the
multiplicator representation into an explicit list of powers of two, and b)
hybrid bit-parallel multplicand/bit-serial multiplicator processing units. PRA
exploits two sources of ineffectual computations: 1) the aforementioned zero
product terms which are the result of the lack of explicitness in the
multiplicator representation, and 2) the excess in the representation precision
used for both multiplicants and multiplicators, e.g., [2]. Measurements
demonstrate that for the convolutional layers, a straightforward variant of PRA
improves performance by 2.6x over the DaDiaNao (DaDN) accelerator [3] and by
1.4x over STR [4]. Similarly, PRA improves energy efficiency by 28% and 10% on
average compared to DaDN and STR. An improved cross lane synchronication scheme
boosts performance improvements to 3.1x over DaDN. Finally, Pragmatic benefits
persist even with an 8-bit quantized representation [5].
