Development of Quantized DNN Library for Exact Hardware Emulation
  Quantization is used to speed up execution time and save power when runnning
Deep neural networks (DNNs) on edge devices like AI chips. To investigate the
effect of quantization, we need performing inference after quantizing the
weights of DNN with 32-bit floating-point precision by a some bit width, and
then quantizing them back to 32-bit floating-point precision. This is because
the DNN library can only handle floating-point numbers. However, the accuracy
of the emulation does not provide accurate precision. We need accurate
precision to detect overflow in MAC operations or to verify the operation on
edge de vices. We have developed PyParch, a DNN library that executes quantized
DNNs (QNNs) with exactly the same be havior as hardware. In this paper, we
describe a new proposal and implementation of PyParch. As a result of the
evaluation, the accuracy of QNNs with arbitrary bit widths can be estimated for
la rge and complex DNNs such as YOLOv5, and the overflow can be detected. We
evaluated the overhead of the emulation time and found that it was 5.6 times
slower for QNN and 42
  times slower for QNN with overflow detection compared to the normal DNN
execution time.
