Performance-Efficiency Trade-off of Low-Precision Numerical Formats in
  Deep Neural Networks
  Deep neural networks (DNNs) have been demonstrated as effective prognostic
models across various domains, e.g. natural language processing, computer
vision, and genomics. However, modern-day DNNs demand high compute and memory
storage for executing any reasonably complex task. To optimize the inference
time and alleviate the power consumption of these networks, DNN accelerators
with low-precision representations of data and DNN parameters are being
actively studied. An interesting research question is in how low-precision
networks can be ported to edge-devices with similar performance as
high-precision networks. In this work, we employ the fixed-point, floating
point, and posit numerical formats at $\leq$8-bit precision within a DNN
accelerator, Deep Positron, with exact multiply-and-accumulate (EMAC) units for
inference. A unified analysis quantifies the trade-offs between overall network
efficiency and performance across five classification tasks. Our results
indicate that posits are a natural fit for DNN inference, outperforming at
$\leq$8-bit precision, and can be realized with competitive resource
requirements relative to those of floating point.
