NullaNet Tiny: Ultra-low-latency DNN Inference Through Fixed-function
  Combinational Logic
  While there is a large body of research on efficient processing of deep
neural networks (DNNs), ultra-low-latency realization of these models for
applications with stringent, sub-microsecond latency requirements continues to
be an unresolved, challenging problem. Field-programmable gate array
(FPGA)-based DNN accelerators are gaining traction as a serious contender to
replace graphics processing unit/central processing unit-based platforms
considering their performance, flexibility, and energy efficiency. This paper
presents NullaNet Tiny, an across-the-stack design and optimization framework
for constructing resource and energy-efficient, ultra-low-latency FPGA-based
neural network accelerators. The key idea is to replace expensive operations
required to compute various filter/neuron functions in a DNN with Boolean logic
expressions that are mapped to the native look-up tables (LUTs) of the FPGA
device (examples of such operations are multiply-and-accumulate and batch
normalization). At about the same level of classification accuracy, compared to
Xilinx's LogicNets, our design achieves 2.36$\times$ lower latency and
24.42$\times$ lower LUT utilization.
