{'fraction_non_alphanumeric': 0.027896486229819564, 'fraction_numerical': 0.008784425451092117, 'mean_word_length': 4.924753867791843, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
FPGA Realization of Hyperbolic Function

Machine Learning (ML) is applied in many real world applications. Artificial Neuron (AN) is the main building block of Artificial Neural Networks (ANN) which is the backbone of ML. ML algorithms for different applications like image and speech recognition etc., speech with large data are implemented on workstations with GPUs. However, the power of ML techniques for real time applications like monitoring, control and diagnostics, typical in power and process plant is still in nascent stage. Most of these applications require FPGA realization of ANNs. Realization of AN with both sigmoid and hyperbolic tangent activation functions along with results is reported in this paper. Design of building blocks consisting of Floating Point Unit (FPU) and Function approximation (FA) are presented. Implementation using Verilog HDL along with results of accuracy with simulated inputs and gate count and timing diagrams are presented.


I. INTRODUCTION
AND, OR and NOT. Focus of Jeremie Detrey et al [6] is on implementing floating-point elementary functions on FPGAs which is contemporary problem. They report trigonometric function approximation based on last bit accuracy.
Derek Nowrouzezahrai et al [7] has presented a computationally efficient algorithm for cosine function based on Taylor's series approximation along with simulation results on Altera Stratix II FPGA. Series approximation is truncated based on accuracy in the last bit weight of the mantissas. P. Graf [8] designed and implemented a VLSI circuit for neural network for classification, particularly for image processing application, consisting of an array of 54 amplifiers with inputs and outputs interconnected through a matrix of resistive elements. All of the coupling elements are programmable resistive connection which can be turned on or off. Ranjeet Ranade [9] et al describes a technique to realize a novel digital multiplier used in Artificial Neural Network (ANN) and studied a generalized `Energy Function' for multiplier and its hardware realization by combining conventional digital hardware with a Neural Network. The design of Neurons, and the digital multiplier are described in this paper along with the simulation results. Valeriu Beiu [10] proposed a method to compute sigmoid function and its derivative in digital hardware by sum of steps and suggested that such algorithms are area-efficient. B.K. Bose [11] research was one of the earliest in floating point operation realizations in VLSI. Focus of research world over in the area of VLSI realization of ANN is on (a) efficient realization of floating point operations with real numbers (b) realization of efficient and accurate representation of functions (c) realization of artificial neuron and the feed forward ANN. Real time realization of the neural net with feed forward mode in FPGA is of recent interest to researchers. Both structural and behavioral models can be considered for implementation of ANN in FPGA. Behavioral Models and realization of an AN with different activation functions along with a floating point algebraic operator for real valued inputs, in FPGA environment is reported in this paper.

II. ARTIFICIAL NEURON
Fundamental building block of Artificial Networks is Artificial Neuron, also called as Perceptron. Figure 1 describes a typical AN.
Rapid progress in Machine Learning in the last decade is opening opportunities for application of Deep Neural Networks and Machine Learning in real time control, monitoring and diagnostics of power and process applications. Enthalpy based real time control and sensor data reconciliation of power plants require repeated usage of steam and gas properties along with their Jacobian and Hessians. Steam and gas properties are, generally, represented in the form of complex Gibbs free energy or Helmholtz free energy equations. The properties are represented at discrete values and obtaining the function values and their derivatives is carried out by repeated calling of the above complex functions. This is computationally prohibitive. Krishna Dutt et al [1,2] reported a compact ANN representation of these properties for simultaneous obtaining of the properties and their derivatives along with their application to real time data reconciliation of power plant sensor data. Arash Ardakani et al [3] proposed an integer form of stochastic computation for an efficient implementation of a Deep Neural Network based on integral stochastic computing. The proposed architecture was demonstrated on a Virtex7 FPGA, resulting in 45% and 62% average reductions in area and latency. K.P. Sridhar et al [4] studied a testing method for VLSI based single neuron architecture with multiple inputs and one output for bench marking against ISCAS85-C17 circuit. Experimental results were reported on XILINX Spartan III FPGA. Bapuray.D. Yammenavar eta al [5] demonstrated an analog VLSI implementation with memory refresher circuits of a neural network; the design is adopted for digital operations like  AN structure consists of m inputs and their associated weights, a summer of weighted inputs and a mapping function that transforms the sum to an output. This function can be any differentiable function. Sigmoid, Tanh, ReLU are some of the widely used functions. Figure 2 shows the output of these two functions i.e., tanh(x) and sigmoid.
Equation (1)  (1) y= f(Z) (2) In general, an AN has a multivariate input (xi), weight vector and a mapping function. A summer block which (Z) a weight sum y the input vector (xi). The mapping function (f) transforms the weighted input to the output (y). The mapping function can be any analytic function of at least first order continuity. The inputs (x) are normalized to be within (-1,1) or (0,1) band an application, however, the weights can be any real number. The output of y is obtained to be within (0,1) or (-1,1) for sigmoid or tanh respectively. There are many other possible functions of which ReLu has become very popular, particularly for dealing with binary inputs, where the above two functions are useful in case of real valued inputs. ReLu is generally used in deep beleif networks. Sigmoid is more popularly used in multi layer neural networks (MLP) or deep neural networks. The structure on learning functions of an AN is based on studies of biological neuron focus in mammalian brains. These biological neurons reveal that learning is either discriminative or generative in nature. A single AN tries to emulate these two capabilities through the learning function.

III. FLOATING POINT UNIT
Floating point unit is defined as math coprocessor.  Figure 9 shows the summary details of device utilization, in which highest utilization is on number of slots (slices) to the extent of 35%, which can be considered reasonable. Common to both equations (3) and (4) is the function block exponentiation of the variable z. This is approximated as Equation 3(a), approximates the exponentiation with a maximum error of 0.04% at the maximum input range. This is considered reasonable approximation for AN. Based on this approximation, equations (1) and (2) are realized in FPGA as shown in figures 5 and 6, respectively.      Table 1 gives the details of Floating point unit, Sigmoid expression and tanh expression in terms of gate count (Number of Slices), CPU time and Power usage (in terms of watts).
All the blocks are implemented on FPGA with the respective specifications. Family -Spartan 3 Device -XC3S400 Package -PQ208 Speed --4/-5

VI. CONCLUSIONS
Realization of Hyperbolic Function in FPGA is demonstration through Verilog code in Xilinx, ISE 14.7, and simulation tool. Popularly known two activation functions, generally used with real number feature vector, are realized with an overall device utilization of 35%. The design is based on behavioral model. However, it is also possible to realize the Artificial Neuron based on structural model for improving the efficiency of device utilization. The function approximation is reported with an accuracy of 0.5. With an overall cycle time 0.41 seconds with a clock period of 100 MHz. The work reported is considered as building block for realizing a deep neural network of any number of hidden layers with each layer having a finite number of Artificial Neurons. The work can easily be extended to cover other activations functions like ReLU, used in deep belief networks.