{'fraction_non_alphanumeric': 0.022073921971252568, 'fraction_numerical': 0.006673511293634497, 'mean_word_length': 5.068980863373387, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Comparison of Update and Genetic Training Algorithms in a Memristor Crossbar Perceptron

Memristor-based computer architectures are becoming more attractive as a possible choice of hardware for the implementation of neural networks. However, at present, memristor technologies are susceptible to a variety of failure modes, a serious concern in any application where regular access to the hardware may not be expected or even possible. In this study, we investigate whether certain training algorithms may be more resilient to particular hardware failure modes, and therefore more suitable for use in those applications. We implement two training algorithms -- a local update scheme and a genetic algorithm -- in a simulated memristor crossbar, and compare their ability to train for a simple image classification task as an increasing number of memristors fail to adjust their conductance. We demonstrate that there is a clear distinction between the two algorithms in several measures of the rate of failure to train.

circuits. Such devices are capable of performing the computational operations traditionally executed on transistors with a von Neumann architecture, 1-3 while also allowing the development of novel structures suitable for non-traditional methods of computation. For example, with the "CrossNet" described in Ref. 4, one may readily implement a perceptron network in a memristor crossbar by encoding the input patterns as voltages across the rows of the crossbar and summing the resultant currents along the columns, functionally using Ohm's Law to perform the inner products of the perceptron. The output currents may then be passed through CMOS support circuitry to determine the appropriate update scheme for the conductance values of the memristors for the next iteration of the algorithm. 5 Memristor crossbars have also been used to find solutions to partial differential equations by implementing a variety of numerical methods on the crossbar. 3 Such devices and methods are part of the "neuromorphic" computing paradigm, in which the von Neumann architecture that has been the dominant paradigm since the advent of modern computation is replaced by an architecture intended to mimic the function of biological neural networks, trading raw speed for greater connectivity of the basic elements of computation. 6,7 However, memristors are susceptible to defects in ways different from conventional semiconductor circuits. Individual memristors in a crossbar may become "stuck" in the lowresistance state after the application of many voltage pulse update cycles, 8 and improper doping can also cause failure to switch between on and off states. 9 Here we investigate the effects of memristor failure in a memristor perceptron, a simple neural network described in Refs. 4-6.
Though it has been shown that intentional removal of particular nodes can be of benefit to network training, 10 the influence of uncontrolled node failure is less clear. Certainly, a crossbar with all memristors damaged will fail to train. In this study, we are interested in the rate of failure to train in the device, within "reasonable time", with increasing damage. It can be expected that, as the number of damaged memristors increases, the performance of the perceptron will deteriorate rapidly, and we are furthermore interested to know if a different algorithm, implemented on the same device, will exhibit a significantly different failure curve.
There have been many schemes proposed and utilized for training neural networks, 11,12 and many of these have been implemented on crossbars. 5,13 One broad distinction to be made within these schemes is between methods that search the network's configuration space locally, such as gradient descent methods, 14 and those that search globally, such as genetic algorithms.
Genetic algorithms are often used in training neutral networks [15][16][17] and also have been suggested for use in neuromorphic hardware. 18, 19 Although these may be more difficult to implement in 3 memristor crossbars, it is generally assumed that genetic algorithms can potentially provide better convergence in situations where other methods may get stuck in local minima. It is not clear if local or global schemes will be more suitable for neuromorphic hardware such as memristor crossbars, in which failure of the nodes may occur. In this study, we begin addressing this question through a comparison of an update method (Ref. 5) as an example of the local schemes and a simple genetic algorithm representing the global schemes.
In order to investigate the question above, both update and genetic training algorithms are implemented in a simulation of a memristor crossbar, shown in Fig. 1, and trained on a simple image classification task similar to that in Ref. 5. With the expectation that a small input pattern (and therefore a small crossbar) would decay too rapidly to provide useful data, the size of the training patterns is increased from 3×3 pixels in Ref.
and the decrease has the form where vset and vreset are random values chosen between [1,5.5] to simulate variation in switching threshold and Gmax and Gmin represent the physical conductance bounds of the devices.
Broadly, a genetic algorithm mimics biological evolutionary processes by randomly generating a number of possible solutions, called chromosomes, and evaluating their performance against a certain measure of fitness. Those that perform well are subjected to some degree of further randomization, or mutation, and then "bred" together in a process that mixes some amount of the "parent" chromosomes together in offspring chromosomes. This new set of proposed solutions, both parents and offspring, is again evaluated for fitness, and the process is repeated until an acceptable solution is found. A single crossbar can hold only one chromosome at a time, which requires the genetic algorithm's search to be performed sequentially. This raises the question of how to store the non-active chromosomes. Since part of the neural network's processing must be handled by the CMOS support circuitry, the conductance values of that active chromosome can be read by small "read" voltage pulses and stored on that same support circuitry.
Then, when necessary, the chromosomes can be switched out by the application of appropriate "write" voltage pulses across the crossbar. 4,20,21 Our genetic training algorithm first generates random conductance matrices as its chromosomes, corresponding to crossbars with random initial memristor values. The matrices' conductance values are altered or swapped according to the algorithm's rules, which in a physical device would be achieved by voltage pulses applied to the appropriate device, much like the update scheme. The genetic scheme also follows a similar algorithm to the update scheme to produce synaptic outputs. Specifically, given the nth conductance matrix ) with elements /0 ) and training vector ̅ ( with elements / ( , we calculate the ith element of the output vector 2 ( , with elements / ( , as follows, corresponding to the product of matrix G with ̅ ( : Physically, these outputs may be interpreted as the currents resulting from voltage pulses applied to the inputs of the crossbar, determined by the training patterns. Each output vector 2 ( is then passed, element-wise, through an activation function, producing output vector ̅ ( with elements: where β is a scaling factor of 2×10 5 A −1 to center the argument around 1. These synaptic outputs for each training pattern, calculated identically to those in the update method, are then used to find the cost of each chromosome, namely the mean-square of the error of each training vector's response from the ideal output for that letter category: where the error of the ith training pattern's output from the ideal, ( ), is and the element-wise difference ( ), between an output vector corresponding to the mth category of letter and the ideal is The magnitude of the ideal output vector is chosen simply to be large relative to the size of a typical output value. However, as the damage grows more extensively, the average number of matrices generated before training is achieved steadily increases, as seen in Fig. 3a and Fig. 3b, and a few trials begin to fail entirely. For the update method, the weight matrices in such trials typically feature several memristors that had capped out at the physically imposed clipping values of 10 μS and 100 μS.
As the number of broken memristors increases, those memristors which could still be updated become increasingly saturated with the capped values, until eventually all 100 trials consistently fail for each number of broken memristors, as seen in Fig. 4a. The update method demonstrates clear superiority to our simple genetic algorithm in the face of increasing damage. The genetic algorithm begins to suffer a rapid increase in failure about 75 memristors earlier than the update scheme, as shown in Fig. 3c, and the update method maintains more rapid and more reliable training throughout. Though it is interesting to note the similarity between the tail end of the failure curves in Fig. 3, we have shown conclusively that, particularly in the early phase of increasing device failure, there is a definite advantage to using one training algorithm over another in the context of device failure. Given these results, the obvious next question is to determine under which circumstances certain algorithms might respond better to damage than others. Certainly, we might expect that a better-designed genetic 9 algorithm could outperform the update scheme used in this report, but can we find some situation in which a particular class of algorithm is superior to all others for this simple classification task?
Perhaps a failure mode other than the one used in this paper could evoke this behavior.
Furthermore, what of more advanced machine learning tasks and methods? As discussed in Ref.
4, the CrossNet can be applied to a greater range of machine learning tasks, especially those that are well-suited for a neural net architecture. It may be interesting to explore if different algorithms might exhibit superior resilience under failure for such tasks. Fig. 4b indicates that, at least for this choice of task and algorithms, the ability of the network to generalize from the training data, which is one of the most pertinent metrics of a network's performance, differs little between the algorithms under increasing damage. A potential avenue for future research is further investigation of this aspect of performance, testing if algorithms might be found that do perform better in this regard.
Though this work does not initially aim to investigate the classification performance of the networks, the behavior displayed in Fig. 5 raises an interesting question for future work. In both Fig. 5a and Fig. 5b, the classification performance of the genetic algorithm degrades sooner with increasing failure than the update scheme, in both training and testing. This suggests that it might be interesting to also investigate classification for various training algorithms, and this might be further extended with testing under different failure modes. Another line of future research could be comparing the algorithms under failure modes that a memristor device might experience other than becoming stuck at a particular random value. It is intriguing that the update scheme performs better than the genetic algorithm when the percentage of broken memristors is between 50% and 80%, as shown in both the average number of failed trials (Fig. 3c) and the classification performance (Fig. 5). Here we present some tentative explanations. First, if we consider the perceptron's energy landscape, we can hypothesize that as more nodes get "stuck", the dimension of the configuration space reduces, making the landscape "simpler", therefore diminishing the advantage of the genetic algorithm in escaping the local minima. Secondly, since it does not use the local properties of the energy landscape, the genetic algorithm is not as efficient as the update method in optimizing the resistance values of non-damaged memristors. We can see from Fig. 4a that when the percentage of damaged memristors exceeds 50%, the fraction of non-damaged memristors set to the extreme values increases rapidly in the update scheme, while their fraction in the genetic algorithm scheme only rapidly increases after 80% of memristors are damaged. Notably, this range (50% to 80%) is also where we observe better performance from the update scheme. We may hypothesize that an effective way for a perceptron to accommodate the damaged nodes is by pushing the nondamaged nodes to their extreme values. Although more studies are needed, these two hypotheses nevertheless suggest that local schemes may be more beneficial for mitigating accumulated failures in hardware-based neuromorphic systems where the range of parameters in each node are physically constrained.
In summary, we implement the local update method and a genetic algorithm in a simulated memristor crossbar perceptron and compare their behaviors in image classification as the number of damaged memristors increases. We find that the update method is more resilient to the accumulation of failed memristors and exhibits an advantage in classification performance.
Further investigations on the robustness of training algorithms in memristor crossbars against device failures are suggested.

ACKNOWLEDGMENTS
The work was supported by the Faculty Research Grant from the College of Arts and Sciences, the University of Memphis.