TFApprox: Towards a Fast Emulation of DNN Approximate Hardware
  Accelerators on GPU
  Energy efficiency of hardware accelerators of deep neural networks (DNN) can
be improved by introducing approximate arithmetic circuits. In order to
quantify the error introduced by using these circuits and avoid the expensive
hardware prototyping, a software emulator of the DNN accelerator is usually
executed on CPU or GPU. However, this emulation is typically two or three
orders of magnitude slower than a software DNN implementation running on CPU or
GPU and operating with standard floating point arithmetic instructions and
common DNN libraries. The reason is that there is no hardware support for
approximate arithmetic operations on common CPUs and GPUs and these operations
have to be expensively emulated. In order to address this issue, we propose an
efficient emulation method for approximate circuits utilized in a given DNN
accelerator which is emulated on GPU. All relevant approximate circuits are
implemented as look-up tables and accessed through a texture memory mechanism
of CUDA capable GPUs. We exploit the fact that the texture memory is optimized
for irregular read-only access and in some GPU architectures is even
implemented as a dedicated cache. This technique allowed us to reduce the
inference time of the emulated DNN accelerator approximately 200 times with
respect to an optimized CPU version on complex DNNs such as ResNet. The
proposed approach extends the TensorFlow library and is available online at
https://github.com/ehw-fit/tf-approximate.
