{'fraction_non_alphanumeric': 0.0297920892494929, 'fraction_numerical': 0.010775862068965518, 'mean_word_length': 4.971612414837245, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Using Libraries of Approximate Circuits in Design of Hardware Accelerators of Deep Neural Networks

Approximate circuits have been developed to provide good tradeoffs between power consumption and quality of service in error resilient applications such as hardware accelerators of deep neural networks (DNN). In order to accelerate the approximate circuit design process and to support a fair benchmarking of circuit approximation methods, libraries of approximate circuits have been introduced. For example, EvoApprox8b contains hundreds of 8-bit approximate adders and multipliers. By means of genetic programming we generated an extended version of the library in which thousands of 8- to 128-bit approximate arithmetic circuits are included. These circuits form Pareto fronts with respect to several error metrics, power consumption and other circuit parameters. In our case study we show how a large set of approximate multipliers can be used to perform a resilience analysis of a hardware accelerator of ResNet DNN and to select the most suitable approximate multiplier for a given application. Results are reported for various instances of the ResNet DNN trained on CIFAR-10 benchmark problem.


I. INTRODUCTION
Many computationally intensive applications (such as image recognition, video processing and data mining) feature an intrinsic error-resilience property [1]. Since they often process noisy or redundant data and their users are willing to accept certain errors in many cases, the principles of approximate computing can be employed in the design of their energyefficient implementations [2]. At the circuit level, approximations (i.e. circuit simplifications) are intentionally introduced to find a good trade-off between power consumption, performance and error. A distinguished class of applications among all these error resilient applications are hardware accelerators of deep neural networks (DNNs) [3]. In the case of DNNs, approximate implementations have been proposed at the level of DNN architecture, data representation, arithmetic operations, memory access and memory cells [3]- [5].
The approximations can be introduced to the circuit in various steps of the standard circuit design flow. In this work, we primarily focus on the technology independent logic synthesis step. The approximations introduced in this step, the so-called functional approximations, modify the Boolean function of the circuit. It has one important advantage -the approximate circuit can be implemented in arbitrary ASIC as well as FPGA technology because it is assumed that the technology dependent implementation is performed by means of common open source or commercial tools after the approximation is conducted.
The methods introduced for the functional approximations can be divided into two categories: (1) manual, and (2) automated. The manual (ad-hoc) methods have been developed for specific circuit components such as adders and multipliers [6], [7]. On the other hand, the automated methods use some general-purpose circuit simplification, resynthesis and approximation techniques and enable us to approximate arbitrary circuits. These methods start with an original (exact) circuit and, typically iteratively, modify its structure.
However, the functional approximation of complex circuits is a time-consuming process. As many of these circuits contain common arithmetic components (circuits) such as adders and multipliers, they can be approximated by replacing selected components by their approximate implementations available in a suitable library. A comprehensive library of approximate arithmetic circuits was introduced in 2017, see EvoApprox8b in [8]. These circuits were designed by means an automated approximation algorithm based on genetic programming, which will be described in Section II. EvoApprox8b contains hundreds of 8-bit approximate adders and multipliers.
The goal of this paper is to extend this library to contain more approximate implementations of arithmetic circuits, with the focus on hardware accelerators of DNNs. By means of genetic programming we generated an extended version of the library in which thousands of 8-to 128-bit approximate arithmetic circuits are included. These circuits form Pareto fronts with respect to several error metrics, power consumption and other circuit parameters. In our case study we show how a large set of approximate multipliers can be used to perform a resilience analysis of a hardware accelerator of ResNet DNN and to select the most suitable approximate multiplier for a given application. Results are reported for various instances of the ResNet DNN trained on CIFAR-10 benchmark problem [9].

ARITHMETIC CIRCUITS
The method used to obtain the library follows the methodology introduced in [10]. It is a general-purpose approximation method for combinational circuits based on Cartesian Genetic Programming (CGP). CGP represents candidate circuits as directed acyclic graphs and iteratively modifies these circuits to reach the design objectives while ensuring that various constraints (e.g. the error is below a given threshold) are not violated.

A. Errors of approximate circuits
Selection of the error metrics is the key step of the whole design. The quality of approximate combinational circuits is typically expressed using one or several error metrics, where the most commonly used ones are: the error rate (ER), the mean absolute error (MAE), the mean square error (MSE), the mean relative error (MRE), the worst case error (WCE), the worst case relative error (WCRE), see eq. 1 -6 in which the output of the approximate circuit and original (exact) circuit is O approx and O orig , n i is the number of primary inputs, the operand's width is n i /2 bits and ∀i enumerates all possible input vectors.
B. Cartesian genetic programming CGP particularly differs from other genetic programming branches in (1) the solution representation and (2) the search mechanism [11]. 1) Representation: A candidate circuit is represented as an integer netlist (the so-called chromosome) describing a constant number of nodes (N ). These nodes are organized in a two-dimensional grid of n c columns and n r rows (N = n c · n r ). The number of primary inputs and outputs of the circuit is denoted n i and n o . Each node implements one of the functions specified in the set of functions Γ and has up to n a inputs and a single output. For gate-level circuits, Γ usually contains a set of binary logic functions (n a = 2). Fig. 1 gives an example.
2) Search algorithm: Every candidate circuit represents one design point in the design space. In CGP, new designs are created by introducing small random modifications to the chromosome. This operation is called the mutation and it typically modifies h integers of the chromosome. Note that A two-bit multiplier represented in CGP with parameters: n i = no = 4, nc = nr = 3, na = 2, Γ = {0 identity , 1 not , 2 and , 3 or , 4 xor , 5 nand , 6 nor , 7 xnor , 8 cont0 , 9 const1 }. all modifications must lead to valid circuits, i.e. only valid function codes and connections can be created.
The search method is based on the (1 + λ) evolutionary strategy which is usually used for a single-objective circuit approximation [11]. The search algorithm can start with either a randomly generated initial population or existing designs. The population size is 1 + λ. After evaluating the initial population (i.e. measuring the circuit functionality and cost) the following steps are repeated until the termination condition is not satisfied: (i) the best-scored circuit (called the parent) is selected; (ii) λ offspring circuits are created from the parent by means of mutation; (iii) the population is evaluated.

C. Circuit approximation using CGP
If a single-objective CGP is applied, the target error range (e.g. the worst-case error), determined by e min and e max , is specified by the user. The goal is to minimize the number of gates (or area or power consumption) while the error of the circuits is kept between the target values e min and e max . If various tradeoffs between the error and the number of gates are requested, CGP is executed several times with e max as the control parameter.
The multi-objective CGP allows us to optimize the error and other key circuit parameters (area, delay and power consumption) together in one run. We are primarily interested in approximate circuits belonging to the Pareto front which contains the so-called non-dominated solutions. For example, consider two circuits C1 and C2. Circuit C1 dominates circuit C2 if: (1) C1 is no worse than C2 in all objectives, and (2) C1 is strictly better than C2 in at least one objective.
The design process typically starts with an accurate circuit. Candidate approximate circuits are generated from the original circuit using CGP. As millions of candidate circuits are often generated, the evaluation must be fast. For small circuits, an exhaustive circuit simulation utilizing all possible input vectors can be used. However, for large circuits this approach is not scalable. A possible solution is to employ advanced verification methods, based on e.g. Boolean satisfiability solving or binary decision diagram analysis [12], [13]. The methodology can easily handle arbitrary constraints.

III. LIBRARY OF APPROXIMATE ARITHMETIC CIRCUITS
Similarly to [8] we seeded CGP with conventional implementations of target arithmetic circuits. A typical singleobjective CGP run uses the following parameters: N = k, where k is the number of gates of the original (exact) circuit with n i primary inputs and n o primary outputs, λ = 1, h = 5, Γ contains all 2-input gates and 1 million generations are produced. One CGP run is typically finished within the order of tens of minutes, depending on the circuit complexity. In the fitness function, the error is obtained by applying one of the error metrics (eq. 1 -6) and the cost is estimated as the sum of weighted areas of the gates used in the circuit. At the end of evolution, the best-scored circuit is synthesized and its parameters are determined by a common design tool (we used Synopsys Design Compiler, 45 nm process, V dd =1V).
The new version of the library contains thousands of various arithmetic circuits as shown in Table I. Since the enormous number of circuits makes the selection of the most suitable circuit for a given application difficult, we identified a subset of circuits and used them in our experiments. The selection follows the principles of Pareto optimality with respect to several objectives in which power consumption is compared against EP, MAE, WCE, MSE and MRE metrics. For each of the five subsets of components, ten circuits evenly distributed along the power axis are taken. Parameters of evolved approximate 8-bit multipliers (power vs. MAE) are shown in Fig. 2. The evolved circuits can be seen as the new state-of-the-art solutions as they provide better tradeoffs than the circuits of the original version of the EvoApprox8b library; the blue points (that occupy the Pareto front "power vs. MAE") are clearly better solutions than the red points representing the original circuits of EvoApprox8b. Note that EvoApprox8b was compared with the state of art approximate circuits in a greater detail [8], [10]. Selected approximate circuits and their various parameters can be downloaded from https://ehw.fit.vutbr.cz/evoapproxlib.

IV. RESILIENCE ANALYSIS IN DNN HARDWARE ACCELERATORS
In order to introduce suitable approximations to DNNs, a resilience analysis is conducted prior to any implementation steps. The resilience analysis of DNNs is usually performed by removing some neurons, weights, memory accesses or inserting some noise to neurons [4], [14]. Introducing approximate multipliers to convolutional layers is one of the most preferred approximation techniques [15]. The proposed library allows us to perform the resilience analysis (focused on these multipliers) in a more realistic way than previous methods. As many approximate multipliers are available, we can immediately analyze the impact of their utilization not only on the accuracy of classification, but also on the power consumption reduction.
In our case study, we investigate the impact of approximations introduced to the multipliers used in convolutional layers of ResNet [16] networks (Fig. 3) trained to classify CIFAR-10 using TensorFlow [17]. The smallest ResNet-8 network consists of three stages with n = 1 residual block in each stage. It contains 7 convolutional layers and performs 21 millions multiplications in the inference phase. The classification accuracy drops from 83.42% to 82.85% if a common floating point multiplication is replaced with the 8bit (exact) multiplication. This 8-bit multiplier is considered as a golden solution and all proposed approximations will be compared against it. Note that no retraining is performed after introducing an approximate multiplier.
From the set of 29,911 approximate 8-bit multipliers available in the library, we identified a subset for our experiments in the following way. In order to obtain diverse designs, we selected 10 Pareto optimal multipliers with respect to power and MAE and repeated this selection for other four error metrics. After removing duplicate circuits we ended up with 35 approximate multipliers showing high-quality tradeoffs between power and the five error metrics.
All exact multipliers of a given layer of ResNet-8 were then replaced by one of the approximate multipliers. This process has been repeated for all the layers and all the approximate multipliers, but only one layer was modified and one type of approximate multipliers was used in each experiment. Tensor Flow extension TFApprox [18] enabled us to accelerate the simulation of ResNet networks containing approximate multipliers. Fig. 4 shows that the most interesting approximations are obtained if the convolutional layer of the third stage is approximated (see S=3, R=1, C=1 in Fig. 4). As this layer contains 28.2% of all the multipliers, it should undergo the approximation with the highest priority. Introducing the approximate multipliers to the first layer (consisting of 2.09% multipliers) makes a negligible contribution. Table II provides a detailed characterization of selected 8bit approximate multipliers and classification accuracy if these multipliers are employed in all convolutional layers of various instances of ResNet and evaluated on CIFAR-10. Evolved approximate multipliers are compared with common highquality approximate multipliers created with truncation and BAM algorithm [7]. One can observe many distinct tradeoffs between classification accuracy and power budget needed for multiplication operations in convolutional layers. This detailed analysis enables the user to identify the best tradeoff for a particular application. For example, if the goal is to reduce power consumption of multipliers to 50% then it makes no sense to use more complex ResNet than ResNet-32 which shows 86.86% accuracy.

Conv2D
Conv2D ReLu Conv2D  approximate circuits needed in approximate implementations of complex applications such as energy-efficient hardware accelerators of DNNs. A subset of approximate 8-bit multipliers was then utilized in resilience analysis of ResNet. For various ResNet networks we obtained many tradeoffs between the classification accuracy and power consumption. This knowledge can be exploited during the hardware implementation of DNN accelerator.
Our future work will be devoted to applying the proposed error resilience analysis approach which is based on the existence of a large library of approximate components in other applications.
This work was supported by Czech Science Foundation project 19-10137S.