Hardware Approximate Techniques for Deep Neural Network Accelerators: A
  Survey
  Deep Neural Networks (DNNs) are very popular because of their high
performance in various cognitive tasks in Machine Learning (ML). Recent
advancements in DNNs have brought beyond human accuracy in many tasks, but at
the cost of high computational complexity. To enable efficient execution of DNN
inference, more and more research works, therefore, exploit the inherent error
resilience of DNNs and employ Approximate Computing (AC) principles to address
the elevated energy demands of DNN accelerators. This article provides a
comprehensive survey and analysis of hardware approximation techniques for DNN
accelerators. First, we analyze the state of the art and by identifying
approximation families, we cluster the respective works with respect to the
approximation type. Next, we analyze the complexity of the performed
evaluations (with respect to the dataset and DNN size) to assess the
efficiency, the potential, and limitations of approximate DNN accelerators.
Moreover, a broad discussion is provided, regarding error metrics that are more
suitable for designing approximate units for DNN accelerators as well as
accuracy recovery approaches that are tailored to DNN inference. Finally, we
present how Approximate Computing for DNN accelerators can go beyond energy
efficiency and address reliability and security issues, as well.
