{'fraction_non_alphanumeric': 0.030095810633101634, 'fraction_numerical': 0.012098440728912268, 'mean_word_length': 4.892406464467567, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
R-DBN: A Resistive Deep Belief Network Architecture Leveraging the Intrinsic Behavior of Probabilistic Devices

A resistive deep belief network (R-DBN) architecture is developed using the physics of nanomagnets to provide a natural hardware representation for individual probabilistic neurons. Probabilistic spin logic devices (p-bits) are modeled to demonstrate a sigmoidal activation function. A hybrid CMOS/spin based weighted array structure is designed to implement a restricted Boltzmann machine (RBM). Device-level simulations based on precise physics relations are used to validate the sigmoidal relation between the output probability of a p-bit and its input currents. Characteristics of the resistive networks and p-bits are modeled in SPICE to perform a circuit-level simulation, which investigates the performance, area, and power consumption tradeoffs of the weighted array. In the application-level simulation, an R-DBN is implemented in MATLAB for digit recognition using the extracted device and circuit behavioral models. The MNIST data set is used to assess the accuracy of the R-DBN using 100 to 5,000 training images for five distinct network topologies. The results indicate that a baseline error rate of 36.8% for a 784x10 R-DBN trained by 100 samples can be reduced to only 3.7% using a 784x800x800x10 R-DBN trained by 5,000 input samples. Finally, power dissipation and accuracy tradeoffs for probabilistic computing mechanisms using resistive devices are identified.


I. INTRODUCTION
HE interrelated fields of machine learning (ML), artificial intelligence, and artificial neural networks (ANN) have grown significantly in previous decades due to the availability of computing systems powerful enough to train and simulate large scale ANNs within reasonable time-scales, as well as the abundance of data available to train such networks in recent years. The resulting research has realized a bevy of ANN architectures that have performed incredible feats including a wide range of classification problems [1], various recognition tasks [2], and defeating a world champion Go player [3].
Most ML techniques in-use today rely on supervised learning, where the systems are trained on patterns with a known desired output, or label, and then the error between the system's output and the desired output is used to update the system with a technique such as backpropagation [4]. However, intelligent biological systems exhibit unsupervised learning whereby statistically correlated input modalities are associated within an internal model used for probabilistic inference and decision making [5]. One interesting class of unsupervised learning approaches that has been extensively researched is the Restricted Boltzmann machine (RBM) [6]. RBMs can be hierarchically organized to realize deep belief networks (DBNs) that have demonstrated unsupervised learning abilities, such as natural language understanding [7]. Most RBM and DBN research has focused on software implementations, which provides flexibility, but requires significant execution time and energy due to large matrix multiplications that are relatively inefficient when implemented on standard Von-Neumann architectures due to the memory-processor bandwidth bottleneck when compared to hardware-based in-memory computing approaches [8]. Thus, research into hardware-based RBM designs has recently sought to alleviate these constraints.
Previous hardware-based RBM implementations have aimed towards overcome software limitations by utilizing FPGAs [9,10], stochastic CMOS [11], and hybrid memristor-CMOS designs [12,13]. FPGA implementations demonstrated RBM speedups of 25-145 over software implementations [9,10], but had significant constraints such as only realizing a single 128x128 RBM per FPGA chip, routing congestion, and clock frequencies limited to 100MHz [10]. The stochastic CMOSbased RBM implementation proposed in [11] leveraged the low-complexity of stochastic CMOS arithmetic to save area and power. However, extremely long bit-stream lengths negate energy savings and lead to very long latencies. Additionally, a significant amount of Linear Feedback Shift Registers (LFSRs) [14] with different seeds were required in order to produce the uncorrelated input and weight bit-streams. In both the FPGA and stochastic CMOS designs, improvements were achieved by implementing parallel Boolean circuits such as multipliers and pseudo-random number generators for probabilistic behavior, which has significant area and energy overheads compared to leveraging the physical behaviors of emerging devices to perform the computation intrinsically. Bojnordi et al. [12] leveraged resistive RAM (RRAM) devices to implement efficient matrix multiplication for weighted products within Boltzmann machine applications, and demonstrated significant speedup of up to 100-fold over single-threaded cores and energy savings of over 10-fold. Similarly, Suri et al. [13] utilized RRAM devices to implement matrix multiplication in addition to leveraging the stochastic RON resistance of HfO RRAM devices to realize a probabilistic activation function, although additional CMOS circuits were required in addition to continually switching the RRAM device in order to obtain stochastic resistance values, which can increase power and latency considerably.
The work presented herein overcomes many of the preceding challenges by utilizing a novel spintronic p-bit device that leverages intrinsic thermal noise within low energy barrier nanomagnets to provide a natural building block for RBMs within a compact and low-energy package. Additionally, various parameters of a hybrid spin-based and CMOS-based weight array structure are investigated for metrics of power dissipation, and error rate using the MNIST digit recognition benchmarks.

II. FUNDAMENTALS OF BOLTZMANN MACHINES
Boltzmann Machines (BM) are a class of recurrent stochastic ANNs with binary nodes whereby each possible state of the network, , has an associated "energy" determined by the undirected connection weights between nodes and the node bias as described by (1), where is the state of node in , is the bias, or intrinsic excitability of node , and is the connection weight between nodes and [15].
Each node in a BM has a probability to be in state '1' according to (2), where σ is the logistic sigmoid function. BMs, when given enough time, will reach a Boltzmann distribution where the probability of the system being in state is found by (3), where could be any possible state of the system. Thus, the system is most likely to be found in states that have the lowest associated energy.
Restricted Boltzmann machines (RBMs) are BMs that are constrained to two fully-connected non-recurrent layers called the visible layer, where salient inputs clamp nodes to output levels of either zero or one, and the hidden layer, where associations between input vectors are learned, as shown in Fig.  1 (a). By enforcing the conditional independence of the visible and hidden layers, unbiased samples from the input can be obtained in one time-step, which enhances the learning process. RBMs can be readily-implemented by a crossbar architecture, as shown in Fig. 1 (b). The most widely used and effective method for training RBMs in an unsupervised manner is contrastive divergence (CD), which is an approximate gradient descent procedure using Gibbs sampling [16]. CD operates in 3 phases as below: 1. Positive Phase: A training input vector, , is applied to the visible layer by clamping the nodes to either 1 or 0 levels, and the hidden layer is sampled, . 2. Negative Phase: by clamping the hidden layer to , the reconstructed input layer is sampled, ′. Then, clamp the visible layer to ′ and sample the hidden layer to obtain ′. 3. Update the weights according to Δ = ( − ′ ′ ), where is the learning rate and W is the weight matrix.
DBNs are realized when additional hidden layers are stacked on top of an RBM as shown in Fig. 1 (c), and can be trained in a very similar way to RBMs. Essentially, training a DBN involves performing CD on the visible layer and the first hidden layer for as many steps as desired, then fixing those weights and moving up a hierarchy as follows. The first hidden layer is now viewed as a visible layer, while the second hidden layer acts as a hidden layer with respect to the CD procedure identified above. Next, another set of CD steps are performed, and then the process is repeated for each additional layer of the DBN.

III. SPIN-BASED BUILDING BLOCK FOR BM
In this section, we provide a detailed description of the p-bit that provides the building block for our proposed spin-based BM architecture. Individual building blocks are interconnected by networks of memristive devices whose resistances can be programmed to provide the desired weights. For instance, in this paper, we will assume that the memristive devices are implemented using the three terminal spin-orbit torque (SOT)driven domain wall motion (DWM) device proposed in [17].
The activation function used in this paper is achieved by a spintronic building block that has been used in the design of probabilistic spin logic devices (p-bits) for a wide variety of Boolean and non-Boolean problems [18][19][20][21]. The basic functionality of the p-bit shown in Fig. 2 [18] is to produce a stochastic output whose steady-state probability is modulated by an input current, in order to generate a sigmoidal activation function. For instance, a high positive input current produces a stochastic output with a high probability of "0", while a high negative input current produces an output with a high probability of being VDD. In the absence of any input current, the device generates either 0 or VDD outputs with roughly equal probability of 0.5 each, as shown in Fig. 3. This device consists of a 3-terminal, spin-Hall driven MTJ [22] that uses a circular, unstable nanomagnet (Δ≪40 kT), whereby its output is amplified by CMOS inverters as shown in Fig. 2. This MTJ with an unstable free layer can be fabricated using standard technology such that the surface anisotropy to achieve perpendicular magnetic anisotropy (PMA) that is not strong enough to overcome the demagnetizing field. Thus, the magnetization stochastically rotates in the plane, due to the presence of thermal fluctuations.

A. Device Operation
The charge current that is injected to the spin-Hall layer creates a spin-current flowing into the circular FM (in the +y direction), which does not have an axis with any preferential geometry. The spin-polarization of this spin-current is in the (±z) direction, and pins the magnetization in the (+z) or (-z) direction depending on the direction of the charge current, through the spin-torque mechanism [22]. The inherent physics of the spin-current driven low-barrier nanomagnet provides a natural sigmoidal function when a long time average of magnetization is taken. Through the tunneling magnetoresistance effect, a charge current flowing through the MTJ with a stable fixed layer detects the modulated magnetization as a voltage change. To achieve this, a small read voltage VR is applied between the V+ and V-terminals through a reference resistance R0, adjusted to the average conductance of the MTJ (R0 -1 =GP+GAP/2) where GP and GAP represented conductance in parallel (P) and anti-parallel (AP) states, respectively. This voltage becomes an input to the CMOS inverters that are biased at the middle point of their DC operating point, creating a stochastic output whose probability can be tuned by the input charge current.

B. Physics-based Modeling
Each component of the device is represented by an independent spin-circuit based on experimentally-benchmarked models that have been established in [23] and simulated as a spin-circuit in a SPICE-like platform. The details of each module and the combined circuit implementation of the device shown in Fig. 2 can be found in [18,23]. Here, we obtain an analytical approximation to the time-averaged behavior of the output characteristics. We start by relating the charge current flowing in the spin Hall layer to the spin-current absorbed by the magnet, assuming short-circuit conditions for simplicity, i.e. 100% spin absorption by the FM: where Is is the spin-current in Amperes, Ic is the charge current, θ is the spin-Hall angle, L, t, λ are the length, thickness and spin diffusion lengths for the spin-Hall layer. The length and width of the GSHE layer are assumed to be the same as the circular nanomagnet. With a suitable choice of the L and t, the spincurrent generated can be greater in magnitude than the charge current generating "gain." For the parameters used in this paper, which are listed in Table I, the gain factor β is ~10. Next, we approximate the behavior of the magnetization as a function of an input spin-current, polarized in the (±z) direction. In the numerical simulation, the time-dependent dynamic response of the circular magnet is simulated by the stochastic Landau-Lifshitz Gilbert (sLLG) equation, however it is possible to approximate the steady-state behavior of magnetization from the Fokker-Planck Equation (FPE), closely following the description outlined in [24]. For a magnet with only a PMA in Fig. 2. Possible implementation of a p-bit: 3-Terminal spin-Hall driven MTJ, utilizing a free layer with a circular (low-barrier) nanomagnet. The MTJ is part of a voltage division circuit that produces a voltage output correlated with the thermal fluctuations of the circular free layer, and CMOS inverters amplify this voltage while providing isolation. Fig. 3. Time-averaged results over 100 ns for p-bit, based on the SPICEbased spin-circuit models. Inset shows the output as a function of time when no input current is present (p=0.5). Analytical fits are based on (7) and (8) for magnetization and probability, respectively.
the ±z direction, a distribution function at steady state can be written analytically as below, as long as the spin-current is also fully in the ±z direction: where Z is a normalization constant, mz is the magnetization along +z,  is the thermal barrier of the nanomagnet, and is is a normalization quantity for the spin-current such that is= Is/(4q/ℏα kT), α being the damping coefficient of the magnet, q the electron charge and ℏ is the reduced Planck constant. It is possible to use (5) to obtain an average magnetization ⟨mz⟩: assuming ≪ kT, (6) can be evaluated to give the Langevin function: which is an exact description for the average magnetization in the presence of a z-directed spin-current for a low barrier PMA magnet. In the present case, however, the nanomagnet has a circular shape with a strong in-plane anisotropy and there is no simple analytical formula that can be derived from a 1D FPE. We use the Langevin function, but with a fitting parameter that adjusts the normalization current is by a factor η, so that the modified normalization constant becomes (4 q/ℏα kT)(η). This factor increases with while elevating the shape anisotropy (Hd~4πMs) and becomes exactly one when there is no shape anisotropy. This method is offered as a new means of fitting the IMA results using the Langevin function that is exact for PMAs while retaining many physical parameters and features that should be useful in the design and understanding of circular magnets. Once the magnetization and charge currents are related by (4)-(6), we can approximate the output probability of the CMOS inverters by a phenomenological equation along with another fitting parameter, χ: which allows us to analytically relate the input charge current to the output probability, with physical parameters. Fig. 3 shows the comparison of the full SPICE-model with respect to (7) and (8) showing good agreement with two fitting parameters η and χ, which fit the magnetization component and the CMOS component, respectively. Figure 4 shows the structure of the weighted array proposed herein to implement the RBM architecture including the SOT-DWM based weighted connections and biases, as well as the pbit based activation functions. Transmission gates (TGs) are utilized in write circuits within the bit cells of the weighted connection to adjust weights by moving the DW position. As investigated by the authors in [25], TGs can provide energyefficient and symmetric switching operation for SOT-based devices, which are desirable during the training phase. Table II lists the required signaling for controlling the training and read operations in the weighted array structure. Herein, a chain of inverters are considered to drive signal lines, in which each successive inverter is twice as large as the previous one.

IV. PROPOSED WEIGHTED ARRAY ARCHITECTURE
During the read operation, write word line (WWL) is The bit line (BL) will be connected to the input signal (VIN), which results in producing a current that affects the output probability of the p-bit device. The direction of the generated current relies on the VIN signal. In particular, since V-is supplied by a voltage source equal to VDD/2, if VIN is connected to VDD the injected current to the p-bit based activation function will have positive value, and if VIN is zero the input current will be negative. The amplitude of the generated current depends on the resistance of the weighted connection which is defined by the position of the DW in the SOT-DWM device. During the training operation, the RWL is connected to GND, which turns OFF the read transistors and disconnects the read path. The WWL is connected to an input pulse (VPULSE) signal which activates the write path for a short period of time. The duration of the VPULSE should be designed in a manner such that it can provide the desired learning rate, , to the training circuit. For instance, a high VPULSE duration results in a significant change in the DW position in each training iteration, which effectively reduces the number of different resistive states that can be realized by the SOT-DWM device. Resistance of the weighted connections can be adjusted by the BL and SL signals, as listed in Table II. A higher resistance leads to a smaller current injected to the p-bit device. Therefore, the input signal connected to the weighted connection will have lower impact on the output probability of the p-bit device, which means the input signal exhibits a lower weight. The bias nodes can also be adjusted similar to the weighted connection.

V. SIMULATION RESULTS
In order to analyze the RBM implementation using the proposed p-bit device and the weighted array structure, we have utilized a hierarchical simulation framework including circuitlevel and application-level simulations. In circuit level simulation, the behavioral models of the p-bit and SOT-DWM devices were leveraged in SPICE circuit simulations using 20nm CMOS technology with 0.9V nominal voltage to validate the functionality of the designed weighted array circuit. In application-level simulation, the results obtained from devicelevel and circuit-level simulations are used to implement a DBN architecture and analyze its behavior in MATLAB.

A. Circuit-level simulation
The device-level simulations shown in Fig. 3 verified a sigmoidal relation between the input current of the p-bit based activation function and its output probability. The shape of the activation function is one of the major factors affecting the performance of the RBM. Therefore, in this section we have provided comprehensive analyses on the impacts of weighted connection resistance and weighted array dimensions on the input currents of the p-bit based activation functions, and the power consumption of the weighted array. Table III lists the range of the activation function input currents for various weighted array dimensions, while the resistance of the SOT-DWM device in parallel state (RP) is constant and equals 1MΩ. The experimental results provided in [17,26] exhibit that an MTJ resistance in the MΩ range can be obtained by increasing the oxide thickness in an MTJ structure. The highest positive and negative currents can be achieved while the weighted connections are in parallel state, i.e. lowest resistance, and all of the input voltages (VIN) are equal to VDD and GND, respectively. The difference between the amplitude of positive and negative currents in a given array size with constant RP is caused by the different pull-down and pull-up strengths in NMOS read transistors. The maximum and minimum output-level "0" probabilities are listed in Table III, which can be obtained according to the measured input currents and the sigmoidal activation function shown in Fig. 3. Moreover, Table IV illustrates the relation between the RP values and input currents of the activation functions, and their corresponding output probabilities, for a given 32×32 weighted array. The lower RP resistance and higher array size provides a wider range of output probabilities which can increase the RBM performance. However, this is achieved at the cost of higher  area and power consumption. The tradeoffs between the array size, weighted connection resistance, and average power consumption in a single read operation is shown in Fig. 5. The lowest power consumption of 22.6 µW is realized by an 8×8 array with RP=1 MΩ. However, this array structure provides the narrowest range of the output probabilities as listed in Table III, which can significantly reduce the performance of the DBN.

B. Application-level simulation
In the application-level simulation, we have leveraged the obtained device and circuit behavioral models to simulate a DBN architecture for digit recognition. In particular, learning rate and the shape of the sigmoid activation function is extracted by the SOT-DWM and p-bit device-level simulations, respectively, while the circuit-level simulations defines the range of the output probabilities. To evaluate the performance of the system, we have modified a MTLAB implementation of DBN by Tanaka and Okutomi [27] and utilized the MNIST data set [28], which includes 60,000 and 10,000 sample images with 28×28 pixels for training and testing operations, respectively. We have used Error rate (ERR) and root-mean-square error (RMSE) metrics to evaluate the performance of the DBN, as expressed by the following relations [27]: where M is the number of output classes, N is the number of input data, NF is the number of false inference, xk is the k-th input data and yk represents its corresponding target output, and F is the inference of the trained DBN.
The simplest model of the DBN that can be implemented for MNIST digit recognition consists 784 nodes in visible layer to handle 28×28 pixels of the input images, and 10 nodes in hidden layer representing the output classes. In this paper, we have simulated and examined various R-DBN topologies, as listed in Table V. Fig. 6 shows the relation between the performance of the mentioned R-DBN topologies, and the number of input training samples ranging from 100 to 5,000, which is obtained using 1,000 test samples. The ERR and RMSE metrics can be improved by enlarging the R-DBN structure through increasing the number of hidden layers, as well as the number of nodes in each layer. This improvement is realized at the cost of larger area and power consumptions. Increasing the input training samples can improve the DBN performance as well, however it will quickly converge due to the limited weight values that can be provided by SOT-DWM based weighted connections. As it is shown in Fig. 6, some random behaviors are observed for networks with smaller sizes that are trained by lower number of training samples, which will be significantly reduced by increasing the number of training samples.
The simulation results exhibit the highest error rate of 36.8% for a 784×10 DBN that is trained by 100 training samples. Meanwhile, the lowest error rate of 3.7% was achieved using a 784×800×800×10 DBN trained by 5,000 input training samples. This illustrates that the recognition error rate can be    DBN1  DBN2  DBN3  DBN4  DBN5  Visible layer  784  784  784  784  784  1 st hidden layer  10  500  800  500  800  2 nd hidden layer  -10  10  500  800  3 rd hidden layer  ---10  decreased by increasing the number of hidden layers, and training samples, which is also realized at the cost of higher area and power overheads.

VI. CONCLUSION
In this paper, we developed a hybrid CMOS/spin-based DBN implementation leveraging p-bit based activation functions, as well as SOT-DWM based weighted connections. First, a p-bit device is utilized to produce a probabilistic output, which can be modulated by an input current. The device-level simulations exhibited a sigmoid relation between the input currents of the p-bit device and its output probability. Next, a SPICE-based behavioral model of the p-bit device was developed to design a weighted array structure for implementing RBM. Circuit-level simulations were performed to examine the proposed weighted array. The results obtained showed that the performance of the array can be improved by enlarging the array size, as well as reducing the resistance of the weighted connections. However, these improvements can be achieved at the cost of increased area and power consumption. For instance, the lowest power dissipation among the examined designs belongs to an 8×8 array with the maximum resistance of 1 MΩ for weighted connections. However, this array structure can only provide the output probabilities ranging from 0.175 to 0.77, which is the narrowest range among the examined designs that consequently results in a DBN implementation with lowest accuracy.
Finally, we have simulated a DBN for digit recognition application in MATLAB based on the extracted device and circuit-level behavioral models. Tradeoffs include the relations between the recognition accuracy of the designed DBN architecture and the number of training samples, which are comparable to conventional hardware implementations. As depicted in the results, the recognition error rate decreased substantially for the first thousand training samples, regardless of the size of the array, while benefits continue through several thousand inputs. However, at least two hidden layers are desirable to achieve suitable error rates. These results from the MNIST data set to establish and generalize tradeoffs impacting resistive computation. Specifically, key tradeoffs have been identified which impact scalability, power dissipation, and accuracy that are applicable to spin-based and other resistive computing devices within neuromorphic architectures.