{'fraction_non_alphanumeric': 0.02929724416890334, 'fraction_numerical': 0.012852984538615659, 'mean_word_length': 4.486570569324899, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Low precision logarithmic number systems: Beyond base-2

Logarithmic number systems (LNS) are used to represent real numbers in many applications using a constant base raised to a fixed-point exponent making its distribution exponential. This greatly simplifies hardware multiply, divide and square root. LNS with base-2 is most common, but in this paper we show that for low-precision LNS the choice of base has a significant impact. We make four main contributions. First, LNS is not closed under addition and subtraction, so the result is approximate. We show that choosing a suitable base can manipulate the distribution to reduce the average error. Second, we show that low-precision LNS addition and subtraction can be implemented efficiently in logic rather than commonly used ROM lookup tables, the complexity of which can be reduced by an appropriate choice of base. A similar effect is shown where the result of arithmetic has greater precision than the input. Third, where input data from external sources is not expected to be in LNS, we can reduce the conversion error by selecting a LNS base to match the expected distribution of the input. Thus, there is no one base which gives the global optimum, and base selection is a trade-off between different factors. Fourth, we show that circuits realized in LNS require lower area and power consumption for short word lengths.

multiplication occur very frequently in systems like neural networks and finite-impulse response (FIR) filters and reduction of its complexity will have a major impact on the overall reduction of area and power. The downside of LNS is that addition and subtraction are non-trivial. A common strategy to implement LNS adders/subtracters use ROM look-up tables, which in the simple case are exponential in the size of LNS word. There is an extensive literature on compacting these tables for large word sizes [6,7,8].
Smaller word sizes can reduce the memory footprint of data, and the complexity of arithmetic for a variety of number systems [9,10,11,12,13,14] which will have a significant impact on the ability to deploy systems in resource constrained embedded devices deployed on the edge of networks. Further fixed point number systems with very short word lengths have been proposed in the literature for a variety of signal processing applications [15,4] while shorter floating and fixed point numbers have also been used for neural networks [16,17,18].
An important aspect of any number system is the distribution of the representable numbers. LNS have a distribution that is non-uniform. Each number is a constant multiple of the previous one, resulting in an exponential distribution. Fully one half of LNS numbers fall between −1 and 1, and representable numbers become increasingly sparse thereafter. In applications like FIR filters, the coefficient space lies within the range of ±1 [4] while the distribution of weights and biases in DNNs is also typically clustered within a small range [19]. Thus, LNS is suitable for use in a number of different applications [20,4,5].
The exponential distribution of LNS depends on two factors: the base and the number of integer/fractional bits in the fixed-point exponent. In existing literature, the base is almost always a power of two, typically 2, √ 2, or 4 √ 2. However, there is no systematic study of different bases in the literature, perhaps because within sensible ranges the choice of base has little impact on LNS with large word size. However, we show that for low-precision LNS the base has a major impact. We make the following contributions.
• We demonstrate that LNS with different bases have different arithmetic rounding errors which is significant for low-precision LNS and can be reduced by selecting an appropriate base.
• We optimize hardware for low-precision LNS arithmetic by selecting a base with favourable add and subtract truth tables. We achieve large savings in circuit area and depth for LNS adder and subtractor circuits by implementing the tables using logic gates as compared to ROM based implementation.
• We show that where input data must be converted from another number format to LNS, base selection can enable a more accurate approximation of the original data distribution, where accuracy is measured by the representation error.
• We also define arithmetic and representation error in a multiplicative sense in the real domain which allows us to describe the error in bits.
• We analyze mixed precision tables where the values stored in addition and subtraction tables have more precision than their inputs and show that realizing these tables results in a reduced error without an exponential increase in the hardware cost to realize them.
• We synthesis adders, subtractors and FIR filters for different word lengths using different number systems and show that circuits LNS based circuits achieve lower area and power consumption as compared to its counterparts for short word lengths.
The paper is organized as follows: Section 2 describes related work. Section 3 gives a brief introduction of logarithmic number system, the effect of base aliasing and of rounding of numbers in LNS. Section 4 discusses the quantization of floating point numbers into LNS. Section 5 describes LNS addition and subtraction using tables and approximation errors in these two operations. The synthesis of addition and subtraction tables is described and evaluated in Section 6 while Section 7 presents a discussion on the trade-offs in selecting the best base based on various parameters. Mixed-precision tables are the focus in Section 8 while Section 9 discusses the hardware cost of realizing LNS adder/subtractor circuits and compared to fixed point multipliers and floating point adders and multipliers. The discussion of hardware cost of real word circuits is extended in Section 10 for FIR filters using the different number systems and word length.

Related Work
LNS has been evaluated as an alternative to fixed and floating point numbers in a number of applications. It provides better dynamic range and round-off noise performance as compared to fixed-point [21] and floating point [22] in some signal processing applications. Paliouras and Stouratis [23] show that LNS can lead to lower switching activity leading to reduced power consumption while also reducing representational error relative to fixed point. Basetas et al. in [24] show that LNS requires a reduced word length when implementing finite impulse response (FIR) and infinite impulse response (IIR) filters. Alam and Gustafsson have also shown [4] that LNS improves the approximation error by around 20% when designing FIR filters as compared to a fixed point design. Coleman proposed a 32-bit logarithmic processor and concludes that LNS provides faster execution, more accuracy and reduced architectural complexity as compared to single precision 32-bit floating point, provided that the adder/subtractor has a latency less than that of a 32 × 32-bit multiplier. Chandra [25] shows an order of magnitude improvement in error-to-signal variances for round-off noise in FIR filters using LNS over floating point number representation.
LNS has also been explored for deep neural networks, especially for very short word lengths. The LogNet inference engine, proposed by Lee et al. [19], uses 4and 5-bit LNS encoding to represent weights and achieves improved results as compared to fixed point encoding without retraining while keeping the activations in floating point. They also encode the activations in LNS with 3and 4-bits while keeping the weights in floating point and show negligible to zero degradation in performance as compared to floating-point. For e.g., the top-5 accuracies (without training) using logarithmic encoding in base-2 with 4 and 5 bits is 73.4% and 74.6%, respectively, for AlexNet and 85.2% and 86.0%, respectively, for VGG16. The corresponding accuracy using 32-bit floating point numbers is 78.3% and 89.8% for the two nets, respectively and much better than corresponding fixed-point performance. This number further improves to a maximum of 85.2$ upon training and logarithmic encoding also has more sparse weights (35.8% against 21.2%) allowing for reduced memory requirements. However, although not clearly stated, the exponent for LNS is represented using integers only and the impact of representing the exponent using fixed-point numbers is not explored.
Miyashita et al. also use an integer representation for the exponent while quantizing using 3and 4-bit encodings for activations and 5-bit for weights [20].  [26], present a low word length based quantization method for LNS to be used in a neural network and have shown LNS to achieve 22.3% lower power as compared to an 8-bit fixed point based design. Arnold et al. [27] present an approach to implement back propagation using tableless LNS ALU based on modified Mitchell's method [28] and achieve one third reduction in the hardware resources as compared to a conventional fixed-point implementation.
However, to the best of authors' knowledge, a comprehensive and systematic study of logarithmic number systems (LNSs) with respect to different bases and error measurement is missing in the literature. Also missing is the study of the impact of choice of base on short word length systems and providing a study of the trade-off involved in selecting bases as a function of the word length while also analyzing the impact of LNS on systems like FIR filters which are heavily dependent on the operations of multiplications and additions. These important aspects form the bulk of the contribution of this work.

Logarithmic Number System (LNS)
A number in the logarithmic number system is represented using the zero-bit, sign-bit and the actual logarithmic value, represented as a triplet (X ), as follows [29]: where z x indicates whether X is zero, s x = sign(X) and m x = log b |X|. The exponent (or logarithmic) value m x is typically represented using two's complement fixed-point representation, with i and f integer and fractional bits (mathematically represented as Q(i, f )) [4,7]. The base-b of an LNS number influences the representation capabilities and complexity of computations and conversions [30]. The real value zero cannot be expressed in the form b m , b = 0, and therefore LNS commonly uses an additional bit to represent zero. Some LNS instead use the most negative possible m x as a special value to represent zero [6].

Bases and Base Aliasing in LNS
The base b is a key parameter of the LNS. For a base b > 1, each LNS value is exponentially larger than the previous one. However, the ratio between successive LNS values is determined not just by the base, but also by the number of fractional bits in the fixed point format Q(i, f ). The least significant bit of the fixed-point exponent has the value 2 −f in the log domain of the LNS number which we define as the unit of least-precision in the log domain (ULP LNS ).
Increasing the log domain value by one ULP LNS corresponds to multiplying the LNS number by b 2 −f in the real domain which we simply define as ULP. We further define an integer radix form, r of the LNS base, where r = b 2 − f . The ratio between successive values of the LNS in the real domain is r.
Note that the LNS with Q(i, f ) and base b contains the same set of values as the LNS with Q(i + f, 0) and base r. Thus, the effect of changing the base of an LNS may be equivalent to simply moving the binary point in the fixed-point representation m x . For example, Miyashita et al. [20] explored the effect of varying the LNS base by experimentally evaluating the bases 2, √ 2 and 4 √ 2. However: In other words, LNS with base-√ 2 and exponent of the form Q(i, f ) contains exactly the same set of values as base-2 LNS with Q(i − 1, f + 1), and base-4 √ 2 with Q(i + 1, f − 1). By selecting the position of the fixed point, standard base-2 LNS can represent implicit bases such as 8 √ 2, 4 √ 2, 2 √ 2, 2, 4, 8, etc. By exploring only bases that can be already represented using base-2, Miyashita et al. [20] investigated only the effect of moving the binary point in m x , not in exploring arbitrary bases. Remarkably, despite a very extensive literature on LNS over several decades, we have been unable to find a statement of this simple identity in the literature.
Thus, when choosing a base for the LNS, the important factors are the total number of bits in m x , that is i + f , and the integer radix form of the base, r. In principle, fixed-point exponents are entirely unnecessary, and we can simply use an integer exponent and a suitable base b = r. However, practical values of r are typically not very convenient numbers to refer to. Therefore, we maintain the existing tradition of representing m x as a fixed-point number, rather than an integer. We explore bases in the range √ 2 to 2 which covers a range of values separated by one binary point position in the format of m x .

Rounding LNS numbers
LNS values have a finite number of digits, and therefore cannot represent all real numbers exactly. Where a real number arising from arithmetic cannot be represented exactly in the LNS, we normally round to a nearby representable value. We focus on rounding to the nearest representable LNS number. Rounding to nearest is a common strategy in floating point number systems; in non-exceptional cases the FP rounding error is bounded by half of one FP mantissa unit of least precision.
A problem with rounding to nearest in LNS is that there are two different and incompatible possible approaches. One option is to round in the LNS domain, i.e., to round the LNS exponent to the nearest representable fixed-point exponent m x . Another option is to round in the real domain, that is to consider the LNS number and its nearest neighbours to the real domain, and choose the rounding that minimizes the real domain error.
The two approaches give slightly different answers. Consider the case of rounding the fixed-point binary exponent value 0.1001 in a base-2 LNS with only integer digits. The choice is between rounding this value downwards to an exponent of 0, or upwards to an exponent of 1. If we round in the log domain, it is clear that 0.1001 is greater than the mid-point of 0.1 and we should therefore round upwards to 1. On the other hand, if we convert these numbers to the real domain, then we are choosing between rounding downwards to 2 0 = 1 or upwards to 2 1 = 2. The value of binary fixed point exponent 0.1001 is 2 9 16 ≈ 1.44043, which is clearly closer to 1 than 2. Thus, if we round in the real domain, we should round this number downwards.
In addition to rounding in the real or log domain, one needs to decide whether to measure absolute or relative error. For fixed point numbers, one either uses the absolute error between the original value and the rounded one or the relative error, as given here: where e a and e r represent the absolute and relative error, respectively.
The measure of error used in literature has typically been the relative error but in the real domain [31], as given by: where e rr is the relative error in the real domain and b is the base. However, recall that in LNS adding one ULP LNS to a number in the log domain corresponds to multiplying the real domain value by the integer radix form, r, of the base, given by b 2 −f and defined as ULP in Section 3.1. Thus, rounding in the log domain minimizes the multiplicative error of the real domain value, that is the factor multiple between the exact real-domain value and the rounded real domain value. On the other hand, rounding in the real domain minimizes the real domain additive relative error, as given in Equation (5), that is the amount that must be added to or subtracted from the exact real domain value to get the rounded real domain value. We define the multiplicative error in the real domain as: where b is the base, as usual. The range of this error, ignoring round to zeros, is given by: where ULP = b 2 −f and ULP h = b 2 −f −1 with b being the base and f being the number of fractional bits. Since this error measure is based on a ratio, it makes more sense to calculate the geometric mean of it, as given by: where e m,i is the individual multiplicative relative error in the real domain given by Equation 6 and n is the total number of values over which the mean is calculated.
The geometric mean corresponds to the exponential of the arithmetic mean of logarithms given by: This implies that calculating the geometric mean in the real domain corresponds to calculating arithmetic mean of the absolute error of the exponents which corresponds to rounding in the log domain.
Furthermore, rounding in the log domain guarantees that in non-exceptional cases the absolute rounding error is bound by half of one ULP LNS . However, a problem arises when rounding to real-domain zero, which has a log domain value roughly equal to −∞. Thus, when rounding a non-zero value to zero, the log-domain error is infinite. If our rounding goal is to minimize the log domain error, we would never round a value to real domain zero. However, never rounding to zero does not make sense, at least to us. Therefore, we round values of less than half of one log domain ULP LNS down to zero.
Rounding in the real domain eliminates the rounding-to-zero anomaly of an infinite error. But rounding to zero causes the real-domain relative errors to be 100%. In all other cases of rounding, the real domain relative error is bound by √ r − 1 for all r > 1. Despite a very extensive literature on LNS over decades, we have not been able to locate a similar discussion of log-domain versus real-domain rounding. In the remainder of the paper we round in the log domain, which allows us to present errors in ULP LNS (which equals 2 −f in the log domain), but rounding in the real domain gives similar results. Furthermore, when presenting results on the average absolute error in the log domain, we report values that underflow to zero -and thus result in an infinite log-domain error -separately from from values that do not underflow.

Conversion of non-LNS inputs
When computing with LNS, it is common that the original inputs to the program are in a different format, such as floating point (FP). We normally convert inputs to LNS once, and perform all subsequent computation in LNS. Any LNS and FP format contain different sets of real values. Thus, when we convert input values to LNS, we normally round each input to the nearest LNS number. Note that the rounding error for converting inputs tend to be cumulatively smaller than arithmetic rounding errors, because most algorithms perform multiple arithmetic operations for each input. In this section we show that we can reduce the conversion rounding error, as compared to base-2 LNS, by selecting another base that better captures the distribution of the input data.
We consider a simple case of input data conversion, where the input consists of a short length FP input type which we convert to a logarithmic number format with the same number of bits. Each FP number has a sign bit, i exponent bits, and f mantissa bits, and we convert to an LNS type with a sign bit, and an exponent with i integer bits and f fractional bits. Very low precision types have been used for both FP and LNS, even as low as 3-bits [20,19]. In light of these contributions it is important to analyze the effect of quantization for shorter word lengths. The word length formats used for our work are given in Table 1.
In our experiments we assume that all FP numbers of the appropriate input type are equally likely to appear. In addition to considering base-2 LNS, we consider around 587 evenly distributed bases between √ 2 and 2. When computing the rounding error in conversion, we use the arithmetic mean of the absolute error of the exponents (in the log domain) as discussed in Section 3.2 and given in Equations 3 and 10.
The problem of rounding a non-zero FP number to zero during conversion is the same as stated earlier in Section 3.2 and we exclude values that underflow or overflow during rounding, as also stated earlier in Section 3.2. Fig. 1(b) shows that for the distribution of FP numbers, the LNS base that gives the lowest quantization error is typically close to 2 for all word lengths given in Table 1.
Note, however, that each LNS base gives both a different range of real-domain values, and a different distribution of those values, with bases close to base-2 having a range closest to a floating point distribution for a given word length. This is reflected in Fig. 1(b) where bases close to base-2 provide the least quantization error across all word lengths. We consider a second experiment where we apply a constant scaling factor to all LNS values, so that the range of LNS values is the same for all bases. With this scaling, different bases lead to different distributions of LNS values, but the range is constant. We use a scaling factor that is the ratio of the maximum floating point and LNS number for each base. Fig. 1(a) shows the effect of keeping the range of LNS values constant through scaling with different bases resulting in distributions that can reduce the quantization error, compared to base-2. For very short word lengths of 5 -7, the optimal base for quantization error is much lower than base-2.
The average absolute representation error, as a percentage of ULP LNS in the log domain and for a scaled LNS distribution, for a few select bases is shown in Table 2 along with the difference of other bases to the best base that gives the minimum representation error, in columns two and three. Other columns describe another type of error which will be explained shortly.
It can be seen from Table 2 that bases other than base-2 can give significant reduction in error, with a minimum 4× improvement for Q(2, 2) over base-2 and a maximum of close to three order of magnitude for larger word lengths. However, the distribution and range of inputs may be domain dependent. A base that results in a distribution of values that is suitable for one domain might be a poor match for another domain. Thus, matching the LNS base to an input distribution may be useful when designing a hardware accelerator that targets a specific domain. But for generalpurpose LNS processing, it is probably not worthwhile to build a base that is specific to one domain into the hardware LNS units. Our experiments show that we can adapt the LNS base to reduce the rounding when converting some input distribution to LNS. However, any reduction in conversion error may also be domain-dependent. In contrast, our approach in the next section improves the accuracy of arithmetic, and is less domain-dependent.

LNS Addition and Subtraction
The In this number system, if we add 1 + 2 = 3, the nearest representable LNS number is 2 (2) ≈ 2.828. Thus, both the set of representable values and the size of rounding errors depends on the base. A common way to compute LNS addition and subtraction uses the identities [6]: where b is the base, S Φ = s x ⊕ s y ⊕ op (op is 0 for addition and 1 for subtraction) and  The challenge of computing function Φ(x) can offset the gains achieved by other useful properties of LNS. A number of methods exist such look-up tables, CORDIC, Taylor series expansion or function approximation [8,6]. We use the look-up table method, which requires O(2 i+f ) space complexity, and is suitable for word lengths up to 20-bits [7]. For the low precision word sizes we consider, in the range 5 − 16 bits, the look-up tables are relatively small.
We represent the Φ + and Φ − functions as look-up tables for different bases. LNS with fixed-point exponent is not closed under addition or subtraction, so we round the values in the tables to the nearest representable LNS number. Different LNS bases lead to number systems containing different sets of numbers. Thus, these look-up tables are different for each base, and therefore the rounding errors are different for each base.
Typically, in an LNS arithmetic unit, look-up tables are implemented as simple read-only memories (ROMs). However, in some cases, the table entries for the two functions are trivial with a very few non-zero values and thus realizing them as logic gates can be beneficial than implementing them as ROMs.

Realization of Large LNS/Small Linear Values in the Subtraction Tables
According to Equation 11, the first entry in the tables correspond to when b mx and b my are equal. This entails two things. First, output of the subtraction of such numbers will be zero and that for φ − (x), the first value corresponds to log b |0|, which is −∞. Similar to this, for the next few values, the difference between b mx and b my is very small implying that subtraction between these two values results in a very small number. This also means that log b |1 − b −x | of Equation 10 returns a very large negative value.
These large negative values in the subtraction table correspond to very small values in the real domain, which are smaller than the smallest representable number. This number corresponds to b −(2 i −2 −f ) . One needs to make a decision about their representation in the tables. In the real domain, values that are smaller than the smallest representable number are either rounded to zero or the smallest representable number, depending on the distance between the number itself and the two adjacent numbers. However, one cannot represent real zero in the LNS domain as that is equivalent to −∞. Thus, the underlying question is the representation of linear zero/LNS −∞ in the subtraction tables.
Since in these cases the output of subtraction either results in a zero or a very small value that is to be rounded to zero in the real domain, we can utilize this by storing zeros in the table and having a small logic circuit that can detect such cases and forcing the output to zero. Storing zeros simplifies the logic circuitry synthesized for realizing the subtraction tables.

Evaluation of Arithmetic Error
We computed the arithmetic error as a percentage of ULP LNS in the log domain (where ULP LNS = 2 −f ) and considered bases ranged from √ 2 and 2, similar to what was done for representation error. The rounding was computed in the log domain with the error measured as defined in Section 3.2.
The result is shown in Fig. 3(a,b) for various word lengths ranging from 5 -16 bits (including one sign bit) where only the base that achieves the minimum arithmetic error for a given word length is shown. The error measures do not include errors generated when values are rounded to zero in the real domain and remain within the bounds given by Equation 7. Also shown in Fig. 3(c,d) is the variation in the average absolute arithmetic error as a percentage of ULP LNS (in the log domain) for Q(4, 3) and that for this particular case, base-1.730 and base-1.802 provide the lowest arithmetic error. This is also indicated in Table 2, where the results for arithmetic error were presented along with representation error. This is a 2% and 7% improvement for the addition and subtraction tables over standard base-2.0.
When rounding to nearest, the average arithmetic error will be around 25% of the ULP LNS with the maximum absolute error being ≤ 0.5× ULP LNS . Figure 3(a,b) shows that base-2 is not the best choice, especially for smaller word lengths.
As mentioned in Section 4, the errors generated due to representation an input distribution is highly domain dependent. However, the values in addition/subtraction tables are dependent on the particular arithmetic operation and the base in which they are represented and this result shows that rounding error for LNS arithmetic can be reduced by selection of an appropriate base.
However, different bases give minimum rounding error for the addition and subtraction tables for different word lengths. These bases do not correspond to the bases which give minimum representation error as shown in Fig. 1. In order to compare the arithmetic and representation error for different interesting bases, Table 2 shows them for each of these bases. It also identifies the best base for each type of error. Also shown are the number of values that are rounded to zero in the real domain. These values are not counted when computing the arithmetic error, for reasons explained in Section 3.2.
Typically, for the fixed point exponent, rounding to nearest results in a maximum error of one half of the ULP LNS and average error of one quarter of the ULP LNS . However, it can be seen in Table 2 that for very short word lengths, the average arithmetic error for bases other than base-2 is much lower than 0.25 × ULP LNS , specially for the subtraction table. For e.g., base-1.417 gives an arithmetic error of 17.1544% of ULP LNS for Q(2, 2) while base 1.415 gives 5 10 15 Total word length  Furthermore, as compared to base-2, significant improvements can be achieved as compared to base-2 of up to 17.6% and 48.2% for addition and subtraction tables, respectively for Q(2, 2). It can also be seen that the representation error for bases which give minimum arithmetic error is typically within the 25% range expected of average rounding error, bar only one exception when the word length specification matches that of half-precision floating point (Q(5, 10)).

Synthesis of Addition and Subtraction Tables
A number of methods exist in literature for realizing Equation 10 in hardware, as mentioned in Section 5. However, since we are dealing with short word lengths, table based approach is the preferred choice [7].
The functions of Φ ± (x)are evaluated by storing pre-computed values in a look-up table and then reading the appropriate value based on the input. The values in the table are quantized to the given word length, various choices for which are listed in Table 1. Typically, look-up tables are realized using read-only ROMs. However, if the table entries are fairly simple, they can be realized using logic gates rather than ROMs. One can also realize some columns of the table using logic gates and others using a ROM depending on the complexity of the columns.
Thus, to realize tables, one has three options: • Synthesize the whole table as a ROM • Synthesize the whole table using logic gates • Synthesize parts of the table using logic gates and parts using a ROM A simple, fixed ROM usually consume only one transistor per bit for data storage [32]. However, even a simple fixed ROM will have additional circuitry as a row/column decoder, sense amplifiers and precharge transistors [33]. A simple differential sense amplifier will consist of five transistors [34] while one needs one precharge transistor per one column of bits. The address decoding for a memory is typically performed by a combination of row and column decoders if memories are arranged as a matrix of words rather than a one dimensional array of individual words or only one address decoder if arranged as a one dimensional array structure.
In the case of realizing the tables using logic gates, one can either synthesize the logic for individual column of bits independently or combine the columns to enable the synthesis tool to share logic to realize various columns of bits.
In our experiments, the total synthesis cost, which we present in terms of transistors, for a combined synthesis of columns of bits is lower than synthesizing individual columns separately. We use the open source ABC synthesis tool [35] and provide the tool with tables described using the Berkeley logic interchange format (BLIF) file [36] and a library containing only NAND, NOR and Inverter gates to enable us to estimate the number of transistors. Each  NAND and NOR gate is typically made up of four transistors while the inverter is made up of two transistors [34]. This can be described as a first degree estimation as we do not take into account the differences in the sizes of the two different types of transistors, the NMOS (N-type Metal Oxide Semiconductor transistor) and PMOS (P-type metal oxide semiconductor transistor) which are used to realize the gates using the complementary MOS technology [34].
However, the question about when to use logic gates and when to use ROM will depend on a threshold. The threshold will be the point when the cost of implementing using gates goes beyond that of ROM which has a constant cost of implementation.
Consider the example of 8-bit LNS which requires only 128 × 7 with the sign bit not stored. If realized as an array structure, the cost for such a ROM will be 896 transistors for the storage, 7 transistors for the precharge, 5 × 7 = 35 transistors for sense amplifiers and 1022 transistors for a 7 × 128 decoder. The implementation cost for the decoder was estimated by synthesizing the decoder using the ABC synthesis tool which was provided a BLIF file describing the truth table of it. This makes for a total cost of 1960 transistors.
However, if we implement the memory as a matrix of 16×8 words, one will need a 4×16 row and 3×8 column decoder which cost 122 and 54 transistors respectively, making the total cost of the memory to be: 896+8×7(precharge)+8× 7 × 5(sense amplifier) + 122 + 54 = 1408 transistors. This number can be further reduced by investigating the values each of the addition and subtraction table takes. For the subtraction table, one needs all 7 bits of output, however, for the addition table, one can discard the two most significant bits (as they are zero) and only store the least significant five bits. This is also evident by Fig. 2 where the range of φ + (x)is limited between zero and one while that of φ − (x)is much larger. This can reduce the total cost for the addition table to 1056 transistors. All of this is summarized in Table 3.
Comparing the cost of ROM with logic gates for realizing complete table shows that for all bases, tables realized using gates has lower area as compared to ROM, as shown in Fig. 4 for 8-bit LNS for all combination of bases between √ 2 and 2, with a minimum savings of 55% and 43% for the addition and subtraction tables, respectively.

Tables using Gates and ROM
Another approach to realize these tables is to realize some columns using logic gates and some using ROM. This may appear beneficial because for those columns with only a few 1s, a gate based realization will result in very small circuits while for the columns where changes in the value are too frequent, a ROM based implementation may be better or of similar complexity as compared to a gate based implementation.
However, experimental evaluation shows that is not the case, as shown in Fig. 5 for select bases which give the minimum and maximum transistor count for each table. The x-axis in each sub-figure shows the number of memory columns where rev. means that columns are combined starting from least significant column while the other alternative shows the starting point is the most significant columns. For the addition table, the two most significant columns are not shown as they are all zeros. Figure 5 shows that the maximum transistor count for the addition table increases from 26 transistors for a single column to 476 transistors for all columns combined using logic gates for base-1.423. This is an improvement of 55% to 93%. Similarly, the savings in transistor count for the subtraction case, in the worst case, is 43% to 91%. Even when combining the columns starting from the most significant column results in lower transistor count as compared to realizing the tables using a ROM.
Thus, considering only the perspective of transistor count, realizing the tables using only logic gates results in a lower transistor count as compared to a simple ROM. Fig. 5 also indicates that for 8-bit LNS, a base of 1.832 and 1.913 gives minimum transistor count for addition and subtraction table, respectively. Thus with a selection of base other than base-2, one can also reduce the implementation cost.

Delay of Logic Circuit for Tables
Typically, for a given word length, the latency of reading a word from a ROM will be the same across all reads, specially if the word being read are in different rows. In a typical implementation, the row address is asserted first to read the whole row (consisting of multiple words) into the sense amplifiers from where the exact word is selected using the column address. The latency of the two decoders (4 × 16 and 3 × 8, as given in Section 6) each for 7and 5-bit memories is 4.7 and 3.3 time units respectively. Only considering the sum of the delay of the two decoders and ignoring the delay associated with the actual reading of the value from transistors, sense amplifiers and any refresh times required, the total delay is 8 time units.
However, the delays associated with realizing the tables as logic gates for different bases and 8-bit output is shown in Fig. 6. The difference between the delay associated with logic gates and ROM based implementation is not significant. With a very significant difference between the transistors required to realize the two alternatives for realizing the addition and subtraction tables, as shown in Figs. 4 and 5, it can be concluded that it is better to use logic gates for realizing the tables for 8-bit LNS.

Deciding the Trade-off for Addition and Subtraction Tables
Four parameters have been discussed in the previous sections. They are: • Representation error • Arithmetic error for φ + (x)and φ − (x) • Area in terms of transistors count for realizing tables using gates for φ + (x)and φ − (x) • Delay for realizing tables using gates for φ + (x)and φ − (x)  For each parameter, there are unique bases in the LNS that gives the corresponding minimum value. The most important of the these parameters are the two types of errors.
The trade-offs involved for one particular word length of Q(3, 3) is shown in Fig. 7 for various bases. Similar tradeoffs exist for other word lengths. All parameters have their importance in the overall design and implementation and each base is optimal in one of the four parameters defined above but none of the base is optimal in all of the four parameters. This figure emphasize our point of the need of analyzing different bases and the priorities while designing and implementing any system. However, it has been clearly shown that base-2.0 is not optimal under different word lengths and for different parameters.

Higher Precision Tables for Very Short Word Lengths
As mentioned earlier, a number of works use very short word lengths for realizing different systems for deployment on edge devices. However, short word lengths can introduce significant precision errors for addition and subtraction. One way to circumvent this is to use tables with higher precision for table values (referred from here on as mixed precision tables for exploratory convenience) and rounding the result only after all required operations for, say, computing the sum of products of data and coefficients in an FIR filter is complete. Thus, there is only one source of arithmetic error at the output (at the cost of higher hardware cost).
The number format given in Table 1 was used for the analysis with fractional word length successively increased up to a maximum of total word length of 16 (including the sign bit) for each case. For e.g., for the case of Q(2, 2), the input  Table 1. word length is 5-bits while the output word length is varied from 5-16 bits. The results for two different total input word length, Q(2,2) and Q (3,3), are shown in Fig. 8, where each value represents the minimum percentage average absolute arithmetic error. For each value, the base that gives this minimum value is unique. As discussed earlier in Section 5.2, for shorter word lengths, the arithmetic error can be reduced much lower than 25% of ULP LNS for bases other than base-2.
For each number format given in Table 1 and all output word lengths, there is no single optimal base which gives the least arithmetic error for various combinations of integer and fractional bits. This phenomenon is further emphasized in Fig. 9 which identifies the various bases which give the least arithmetic error for total output word length. The legend shows the original total word length (without extended fractional bits). For e.g., the plot shows the optimal base for output word lengths ranging from 5-16 bits for the case of Q(2, 2). Again it re-emphasizes the earlier reached conclusion that bases other than base-2 should be considered when designing a system as it can give lower error and implementation cost.

Synthesis of Higher Precision Tables for Very Short Word Lengths
The other key issue to deal with extra fractional bits is the size of these tables, once they are realized in hardware. As shown in Figs. 4 and 5, it is better to realize the tables using gates. However, with increasing output word length for the tables, one needs to analyze whether the conclusion reached in Section 6 still stands. For this, the addition and subtraction tables were synthesized using both ROM and standard cell gates for all input and output word length combinations. Table 4 shows the number of transistors required to realize the ROMs based on the model earlier defined in Section 6. The size of decoders for the various tables were chosen so as to achieve minimum transistor count for their implementation.
However, arithmetic error is more important than transistor count to ensure correctness of result. Although the hardware cost of realizing tables with bases giving minimum arithmetic error is larger than the best case in terms of transistor count, it is still significantly better than realizing it in ROMs. Results for the subtraction table are shown in Table 5 as typically it is more complex to realize than the addition table and shows an order of magnitude improvement, with a minimum reduction of 168% in transistor count to a maximum of nearly 10× reduction.
It can also be seen from Table 5 that as the output word length increases, for each input word length, the transistor count does not increase significantly, making the mixed precision approach an attractive one given the reduced error measurement.  Figure 11: Circuit schematic for LNS addition and subtraction.

LNS Adder and Multiplier Design in Fixed and Floating Point Number System
As identified in Sections 1 and 2, logarithmic number system (LNS) finds application in signal processing and neural networks. Both of these systems are based heavily on the operation of addition and multiplication. The key advantage of LNS over fixed and floating point stem from its inherent simplification of multiplication into addition. However, the drawback with LNS is that it can only approximate the addition operation since addition is not closed under LNS, as outlined in Section 5.
Since multiplication in LNS is reduced to addition, the circuit to realize an LNS multiplier will be similar in complexity to that of a fixed point adder [37]. This is because the exponent of the LNS is typically represented using fixed point. In order to ascertain which number system to use from a hardware complexity purpose it is important to realize the hardware circuit for LNS addition/subtraction and compare it against fixed point multiplier. Since in applications like neural networks the dominant number system used is floating point, it is important to realize floating point addition and multiplication circuits to analyze which number system provides the smallest circuit.
The circuit to perform addition and subtraction in LNS is given in Fig. 11 and is a modified form of the circuit given by Kenny et al in [6]. In the circuit, the functions of φ + (x)and φ − (x)are implemented using look-up tables, as mentioned in Section 6. The zero detect module shown takes into account whether the two numbers being added are both zero (shown by the z x bit of Equation (1)), the difference of two number is zero or so small that the result will be rounded to zero in the real/linear domain, as discussed in Section 5.1. Addition and multiplier circuits were synthesized for logarithmic, fixed and floating point number systems for word length ranging from five to 16-bits using Cadence Genus with the 45nm GSCL liberty cell library from the Free45PDK. For LNS, the bases were varied from √ 2 to 2, similar to how LNS is analyzed in the previous sections. The results are shown in Table 6 where Q(i, f ) indicates i integer and f fractional bits and does not include the sign bit. For fixed point, it needs to be read as Q(i + 1, f ) since 2 ′ s complement number representation is assumed and for floating point, results for both adder (Float.A) and multiplier (Float.M) are presented.
As evident in Table 6 the LNS adder requires lower or similar area as compared to fixed point multipliers for short word lengths up to Q(4, 4). Although the floating point multiplier is the smaller of all circuits (for most of the cases), the LNS multiplier and fixed-point adder will be more simpler and smaller than it, thus overall, floating point will not have the smallest of the circuit. Also important to realize is that for LNS, the same circuit is used to perform both addition and subtraction which opens up the possibility of re-using the same circuit for the two operations if the input sample rate allows for some level of multiplexing. Furthermore, for LNS, the adder and subtractor were realized using the look-up table approach for implementing the functions φ + (x)and φ − (x)in Equation (10). They can also be realized using different techniques like weighted sum of bit-products [6] which can reduce the area complexity. However, in this work all error calculations are based on table based implementations, the same method was used for realizing circuits for consistency.
For each number format, the result for LNS other than base-2 is the base which gives the lowest area. This does not correspond to the shortest delay through the circuit or the lowest power consumption. Fig. 12 gives a better insight into the inherent trade-offs involved here.

Finite-Impulse Response (FIR) Filter: An Application
Frequency selective finite-length impulse response (FIR) filters are used in many applications and are therefore a key building block in a digital signal processing system [38]. The difference equation that defines the output of an N th-order (length N + 1) FIR filter is: where y(n) is the output sequence, x(n) is the input sequence, and h(k) are the coefficients. The direct form of FIR filter to realize Equation (12) is shown in Fig. 13.
The arithmetic complexity of FIR filters primarily depends on the number of multiplications, which according to (12) is proportional to the filter order. The filter order is, for linear-phase single stage FIR (SSF) filters, roughly proportional to the inverse of the width of the transition band [38].
With LNS reducing multiplication to an addition, it finds application in applications like FIR filters where the main operator contributing towards overall hardware complexity is multiplication. However, as stated earlier, addition/subtraction in LNS is non-trivial and requires a table based implementation as shown in Fig. 11. As shown in Section 9, LNS provides lower hardware cost in terms of area for short word length adders/subtractors as compared to its counterparts, it is important to analyze whether these benefits extend to FIR filters.
For this analysis, a 11 th order FIR filter was implemented in hardware using the same setup as described in Section 9 with fixed, floating and logarithmic number systems. The input, output and filter coefficients were assumed to be in respective number systems. For LNS, the FIR filter was realized for a select number of bases identified earlier in Tables 2 and 6. The results of this analysis is shown in Table 7.
The results shown in Table 7 are consistent with those shown in Table 6. FIR filters realized using logarithmic number systems achieve a lower area (minimum savings of 2%) while also having the shortest delay (minimum improvement of 4%)and lower power consumption (minimum improvement of 6%)as compared to its fixed and floating point counterparts up to the word length corresponding to Q(4, 4). Overall, LNS based FIR filters also are the fastest even though occupies an area that is, at a maximum, nearly 3× more than others.  The large area for LNS based FIR filters for larger word lengths is due to the exponential increase in the memory size required for implementing the φ + (x)and φ − (x)functions, as given in Equation 10. This can be reduced by using approaches like the co-transformation approach by Arnold et al. in [39], the multi-table approach proposed by Coleman et al. in [7] or the multipartite table methods proposed by Dinechin et al. in [40].
Typically conversion can be carried out offline and all calculations can be done online in LNS. However, to show the cost of conversion between logarithmic and fixed point, conversion circuits were realized as look-up tables and synthesized using the same process as FIR filter. Only conversion between LNS and fixed point is shown since FIR filters are typically implemented using fixed point. The results are shown in Table 8 and it can be seen that the cost of realizing FIR filters using LNS is still lower than using fixed point, albeit with reduced margins, including conversion results, up to word length of Q(4, 4). Optimized circuits for conversion have been proposed in the literature by Nam et al. in [41], Zhu et al. in [42] and others which can help in reducing this cost.

Conclusion
Logarithmic number system (LNSs) are a valuable alternative to floating point for embedded systems and domainspecific hardware accelerators. Existing LNS research has almost exclusively used base-2. Although other bases, such as base-√ 2 or base-4 √ 2 have been explored, we show that these are simple aliases of base-2, which can be constructed by moving the binary point of the fixed-point exponent in a base-2 LNS.
In this paper we show that non-base-2 LNSs can have significant advantages, particularly for short word lengths from 5 − 16 bits. We first define a numeric error for LNS that measure ULP LNS as an absolute error in the fixed-point exponent of the LNS, which corresponds to a multiplicative error in the corresponding real domain value. We show that errors in conversion to LNS can be reduced dramatically by choosing an appropriate base and scaling factor, at least for the distribution of FP numbers. In our experiments, we reduce the typical conversion error from 0.20 − 0.25 ULP LNS to less than 0.02 ULP LNS .
We note that LNSs with a fixed word size are not closed under addition or subtraction, which leads to rounding errors in these operations. We also show that for a given LNS word size, some LNS bases lead to inherently lower average rounding errors. When rounding to nearest, the maximum arithmetic error for non-special values is 0.5 ULP LNS , and we expect the average to be around 0.25 ULP LNS . For example, in 5-bit (i.e. Q(2,2)) LNS, we show that base-2 leads to average errors of 0.260 (add) and 0.254 (subtract) ULP LNS . In contrast, base-1.417 gives an average error of just 0.227(add) and 0.172 (subtract) ULP LNS . Over millions of operations, even small differences in rounding errors can accumulate to large errors.
Although LNS add/subtract are often implemented with low-latency ROM lookup tables, we show that for small word sizes implementing these functions in dedicated logic is also efficient. The choice of base affects the truth table of the functions computed for LNS add/subtract, and can therefore affect the area and latency of the add/subtract units. By appropriate choice of base, the logic area of the φ + (x)and φ − (x)logic can be reduced by around 2×.
We also analyzed mixed precision tables, where the values stored in the tables were of higher precision than the input. This can reduce the errors for low precision arithmetic. The building blocks of adder, subtractor and multiplier are implemented using logarithmic, fixed and floating point numbers and extended to a complete FIR filters and we showed that LNS can be beneficial from all aspects of area, delay and power for very short word lengths.
We conclude that the choice of base affects the conversion errors, arithmetic errors, and hardware implementation of arithmetic units on several dimensions and that LNS can be used to implement different systems, like the FIR filter. Different bases allow different trade-offs between these goals. Although prior research in LNS has generally assumed base-2, our results show that other bases can reduce arithmetic and conversion errors while simultaneously reducing the area and latency of add/subtract and FIR units. Thus, when designing a low-precision LNS for custom hardware, one should consider looking beyond base-2.