{'fraction_non_alphanumeric': 0.030737731140827947, 'fraction_numerical': 0.010045428370337194, 'mean_word_length': 5.173157437396319, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications

With the advent of dedicated Deep Learning (DL) accelerators and neuromorphic processors, new opportunities are emerging for applying deep and Spiking Neural Network (SNN) algorithms to healthcare and biomedical applications at the edge. This can facilitate the advancement of the medical Internet of Things (IoT) systems and Point of Care (PoC) devices. In this paper, we provide a tutorial describing how various technologies ranging from emerging memristive devices, to established Field Programmable Gate Arrays (FPGAs), and mature Complementary Metal Oxide Semiconductor (CMOS) technology can be used to develop efficient DL accelerators to solve a wide variety of diagnostic, pattern recognition, and signal processing problems in healthcare. Furthermore, we explore how spiking neuromorphic processors can complement their DL counterparts for processing biomedical signals. After providing the required background, we unify the sparsely distributed research on neural network and neuromorphic hardware implementations as applied to the healthcare domain. In addition, we benchmark various hardware platforms by performing a biomedical electromyography (EMG) signal processing task and drawing comparisons among them in terms of inference delay and energy. Finally, we provide our analysis of the field and share a perspective on the advantages, disadvantages, challenges, and opportunities that different accelerators and neuromorphic processors introduce to healthcare and biomedical domains. This paper can serve a large audience, ranging from nanoelectronics researchers, to biomedical and healthcare practitioners in grasping the fundamental interplay between hardware, algorithms, and clinical adoption of these tools, as we shed light on the future of deep networks and spiking neuromorphic processing systems as proponents for driving biomedical circuits and systems forward.

trillion dollars by 2022 [1]. One of the most promising technologies to advance this fast-growing industry is Artificial Intelligence (AI) [2] and its implementation with Deep Learning (DL). DL has shown success in various domains and as its reliability improves, it has pervaded various facets of healthcare from monitoring [3], [4], to prediction [5], diagnosis [6], treatment [7], and prognosis [8], as visualized in Fig. 1(a). The figure shows that the data collected from the patient, which in this case is illustrated as a biomedical signal, but can be any or a combination of other data types such as bio-samples, medical images, temperature, movement, etc. can be processed using a smart DL system that monitors the patient for anomalies and/or to predict diseases. The prediction can inform diagnosis, which itself can benefit from DL algorithms. In addition, DL systems can be used to recommend treatment options and prognosis, which further affect monitoring and prediction in a closed-loop scenario. In every step of this loop, there is a need for a DL training and inference procedure, which requires significant computational resources.
The capacity of AI to meet or exceed the performance of human experts in medical-data analysis [9], [10], [11] can, in part, be attributed to the continued improvement of highperformance computing platforms such as Graphics Processing Units (GPUs) [12] and customized Machine Learning (ML) hardware [13]. These can now process and learn from a large amount of multi-modal heterogeneous general and medical data [14]. This was not readily achievable a decade ago.
Although the DL field has been growing at an astonishing rate in terms of software, algorithms, and architecture developments, its hardware accelerator development currently largely relies on advances by a handful of giant technology companies, most notably Nvidia and its GPUs [15], [16] and Google and its Tensor Processing Units (TPUs) [13], in addition to new startups and research groups developing Application Specific Integrated Circuits (ASICs) for DL training and acceleration. Similarly, while there are significant advances in tailoring deep network models and algorithms for various healthcare and biomedical applications [17], most medical deep networks are currently trained and run on GPUs or in data centers [12], [18]. This mostly requires the use of cloud-based DL processors which rely on costly and power-demanding data centers, as opposed to the effective deployment of DL at the edge on an increasing number of healthcare and medical IoT systems [19] and PoC devices [20], as illustrated in Fig. 1(b). These devices A depiction of (a) the usage of DL in a smart healthcare setting, which typically involves monitoring, prediction, diagnosis, treatment, and prognosis. The various parts of the DL-based healthcare system can run on (b) the three levels of the IoT, i.e. edge devices, edge nodes, and the cloud. However, for healthcare IoT and PoC processing, edge learning and inference is preferred. and systems are desired to be as low-cost, compact, lowpower, and rapid (high throughput) as possible, to facilitate applications at the edge and make smart health monitoring technology more viable and affordable for integration into human life [21]. Furthermore, edge learning and/or inference can enable systems which are mostly independent of the cloud. This feature is critical for highly sensitive medical data and offline operation, which are much desired in healthcare and biomedical settings.
To facilitate at-edge processing, specialized embedded DL accelerators such as Nvidia Jetson and Xavier series [22], as well as Movidius Neural Compute Stick [23], [24] have been produced. These devices and systems have been shown to be quite suitable for healthcare edge or near-edge inference. More recently, examples of specialized embedded hardware systems for medical tasks, such as The Nvidia Clara Embedded, have been proposed. This is a computing platform for edge-enabled AI on the Internet of Medical Things (IoMT). However, as these embedded devices are still relatively power hungry and costly, they are still not ideal learning/inference engines for ambient-assisted healthcare IoT applications and PoC systems. So, there is a need for innovative systems which can satisfy the stringent requirements of healthcare edge devices, to make them available and beneficial to the community at large scales and with affordable costs.
To that end, in this paper, we focus on the use of three various hardware technologies to develop dedicated deep network accelerators which will be discussed from a biomedical and healthcare application point-of-view, even though they could be used for general-purpose smart edge IoT devices. The three technologies that we cover here are CMOS, memristors, and Field Programmable Gate Arrays (FPGAs). It is worth noting that, while our focus is mainly around efficient inference engines at the biomedical application edge, the techniques and hardware advantages discussed here may be also useful for more efficient offline deep network learning, or online on-chip learning. Herein, the term DL 'accelerator' is used for referring to a device that is able to perform DL inference and potentially training.
To provide a self-contained tutorial on the implementation of DL accelerators, we first deliver a brief introduction to the fundamentals of artificial and spiking neural networks and their various architectures. Next, we shed light on why deep networks are power-and resource-hungry and need specific hardware platforms to enable them for edge processing. After that, we discuss recent hardware advances which have led to improvements in training and inference efficiency. These improvements ultimately guide us to more viable edge inference engine options.
When discussing the three target hardware technologies, we show that the field of hardware implementation for customized healthcare and biomedical DL accelerators is very sparse. After reviewing the literature on these DL accelerators, we provide a guided analysis to quantify the performance of various algorithms on different types of DL processors. The results allow us to draw a perspective on the potential future of spike-based neuromorphic processors in the biomedical signal processing domain. Based on our analysis and perspective, we conjecture that for at-edge processing, neuromorphic computing technologies and their underlying Spiking Neural Networks (SNNs) could complement DL inference engines, either through signaling anomalies in the data or acting as 'intelligent always-on watchdogs' which continuously monitor the data being recorded, but only activate further processing stages if and when necessary.
Although there are previous reviews on general AI-based algorithm design and hardware for biomedical applications [25], to the best of our knowledge, this is the first work where a comprehensive tutorial and review is proposed that focuses on customized DL accelerators for biomedical applications. Our contributions that differentiate our work from the available literature can be summarized as follows: • There is no previous comprehensive paper that focuses only on DL accelerators and shows how they can be used for medical and healthcare applications. • Our paper is the first to discuss the use of three different emerging and established hardware technologies for facilitating DL acceleration, with a focus on biomedical applications. • We provide tutorial sections on how one may implement a typical biomedical task on FPGAs or simulate it for deployment on memristive crossbars. • Our paper is the first to discuss how event-based neuromorphic processors can complement DL accelerators for biomedical signal processing. • We provide open-source codes and data to enable the reproduction of our shown results.
We believe these features make our paper a useful contribution to the wider biomedical circuits and systems society with an interest in utilizing mature and emerging technologies and techniques for enabling DL training and inference on the edge of healthcare systems.
The remainder of the paper is organized as follows. In Section II, we define the technical terminology that is used throughout this paper and cover the working principles of artificial and spiking neural networks. We also introduce a biomedical signal processing task for hand-gesture classification, which is used for benchmarking the different technologies and algorithms discussed in this paper. In Section III, we step through the design, simulation, and implementation of Deep Neural Networks (DNNs) using different hardware technologies. We show sample cases of how they have been deployed in healthcare settings. Furthermore, we demonstrate the steps and techniques required to simulate and implement hardware for the benchmark hand-gesture classification task using memristive crossbars and FPGAs.
In Section IV, we provide our perspective on the challenges and opportunities of both DNNs and SNNs for biomedical applications and shed light on the future of spiking neuromorphic hardware technologies in the biomedical domain. Section V presents concluding remarks and discussions.

A. Nomenclature of Neural Network Architectures
Although most DNNs reported in literature are ANNs, DNNs usually refer to more than one hidden layer, independently of whether the architecture is fully connected, convolutional, recurrent, ANN or SNN, or of any other structure. For example, the most widely used DNN type, i.e. a CNN, can be physically implemented as an ANN or SNN, and in both cases it would be 'deep'. However, in this paper, whenever we use the term 'deep', DL, or deep network, we refer to Deep Artificial Neural Networks. For Deep Spiking Neural Networks, we simply use the term SNN.

B. Deep Artificial Neural Networks
Traditional ANNs and their learning strategies that were first developed several decades ago [26] have, in the past several years, demonstrated unprecedented performance in a plethora of challenging tasks which are typically associated with human cognition. These have been applied to medical image diagnosis [27] and medical text processing [28], using DNNs. Fig. 2 illustrates a simplified overview of the structure of some of the most widely-used DNNs. The most conventional form of these architectures is the Multi-Layer Perceptron (MLP). Increasing the number of hidden layers of perceptron cells is widely regarded to improve hierarchical feature extraction which is exploited in various biomedical tasks, such as seizure detection from electroencephalography (EEG) [29]. CNNs introduce convolutional layers, which use spatial filters to learn various parts of the feature space. CNNs also have pooling layers that are placed after convolutional layers to down-sample their outputs to reduce the search space size for subsequent convolutional layers. CNNs have been widely used in medical and healthcare applications, as they are very wellsuited for spatially structured data. Their use in medical image analysis [30] will form a major part of our discussions in this paper. RNNs represent another powerful DL architecture type that has been recently used both individually [31], and in combination with CNNs [32] in biomedical applications. RNNs introduce recurrent cells with a feedback loop, and are especially useful for processing sequential data such as temporal signals and time-series data, e.g. electrocardiography (ECG) [32], and medical text [33]. The feedback loop in recurrent cells gives them a memory of previous steps and builds a dynamic awareness of changes in the input. The most well-known type of RNNs are LSTMs which are designed to mine patterns in data sequences using their short memory of distant events stored in their memory cells. LSTMs have been widely used for processing biomedical signals such as ECG [31], [34]. Although there are many other varieties of DNN architectures, we will focus on these most commonly used types.
1) Automatic hierarchical feature extraction: The above mentioned DNNs learn intricate data features and representations through multiple neural computational layers across various levels of abstraction [35]. The fundamental advantage of DNNs is that they mine the input data features automatically, without the need for human knowledge in their supervised learning loop. This essential feature helps deep networks learn complex features by combining a hierarchy of simpler features learned in their hidden layers [35].
2) Learning algorithms: Learning features from data in a DNN, e.g. the networks shown in Fig. 2, is typically achieved by minimizing a loss function. In most cases, the loss is defined as maximum likelihood using the cross-entropy between training data and the learned model distribution. The loss function minimization happens through optimizing the network parameters (weights and biases). This optimization process minimizes the loss function from the final network layer backward through all the network layers and is therefore, called backpropagation. A typical optimization algorithm that is widely used in DNNs is Stochastic Gradient Descent (SGD) or its several variants [35].
3) Backpropagation in DNNs is computationally expensive: Despite the continual improvement of hardware platforms for running DNNs, training and running these networks remains a highly power consuming and computationally formidable task. The catalyst for the intensive computational requirement, which results in high power consumption, is the feed-forward error backpropagation algorithm, which depends on thousands of epochs of computationally intensive Vector Matrix Multiplication (VMM) operations [26]. These operations, if performed on a conventional von Neumann architecture which has separate memory and processing units, will have a time and power complexity of order O(N 2 ) for multiplying a vector of length N in a matrix of dimensions N × N .
In addition, an artificial neuron in DNNs calculates a sumof-products of its input-weight matrix pairs. For instance, a CNN spatially structures the sum-of-products calculation into a VMM operation. In digital logic, an adder tree can be used to accumulate a large number of values. This, however, becomes problematic in DNNs when one considers the sheer number of elements that must be summed together, as each addition requires one cycle. Table I   This table highlights two key facts. Firstly, MACs are the dominant operation of DNNs. Therefore, hardware implementations of DNNs should strive to parallelize a large number of MACs to perform effectively. Secondly, there are many predetermined weights that must be called from memory. Reducing the energy and time consumed by reading weights from memory provides another opportunity to improve efficiency.
Consequently, significant research has been being conducted to achieve massive parallelism and to reduce memory access in DNN accelerators, using different hardware technologies and platforms as depicted in Fig. 3. Although these goals are towards general DL applications, they can significantly facilitate fast and low-power smart PoC devices [20] and healthcare IoT systems.
In addition to conventional DL accelerators, there have been significant research efforts to utilize biologically plausible SNNs for learning and cognition [36]. Spiking neuromorphic processors have also been used for biomedical signal processing [37], [38], [39]. Below, we provide a brief introduction to SNNs, which will be discussed as a method complementary to DL accelerators for efficient biomedical signal processing later in this paper. We will also perform comparisons among SNNs and DNNs in performing an electromyography (EMG) processing task.

C. Spiking Neural Networks
SNNs are neural networks that typically use Integrateand-Fire neurons to dynamically process temporally varying signals (see Fig. 4(j)). By integrating multiple spikes over time, it is possible to reconstruct an analog value that represents the mean firing rate of the neuron. The mean firing rate is equivalent to the value of the activation function of ANNs. So in the mean firing rate limit, there is an equivalence between ANNs and SNNs. By using spikes as all-or-none digital events ( Fig. 4(i)), SNNs enable the reliable transmission of signals across long distances in electronic systems. In addition, by introducing the temporal dimension, these networks can efficiently encode and process sequential data and temporally changing inputs. SNNs can be efficiently interfaced with eventbased sensors since they only process events as they are generated. An example of such sensors is the Dynamic Vision Sensor (DVS), which is an event-based camera shown in Fig. 4(h). The DVS consists of a logarithmic photo-detector stage followed by an operational transconductance amplifier with a capacitive-divider gain stage, and two comparators. The ON/OFF spikes are generated every time the difference between the current and previous value of the input exceeds a pre-defined threshold. The sign of the difference corresponds to the ON or OFF channel where the spike is produced. This is different to conventional cameras ( Fig. 4(f)), which produce image frames ( Fig. 4(g)). Intuitively, it makes sense to use asynchronous event-based sensor data in asynchronous SNNs, and synchronously generated frames (i.e., all pixels are given at a regular clock interval) through synchronous ANNs. But it is worth noting that conventional frames can be encoded as asynchronous spikes with frequencies that vary based on pixel intensity, and event streams can be integrated over time into synchronously generated time-surfaces [40], [41]. Event-based sensors have been used to process biomedical signals [37], [42] ( Fig. 4(a)), which can be encoded to spike trains ( Fig. 4(b)) to be processed by SNNs or be digitally sampled ( Fig. 4(c)) for use in DNNs for learning and inference ( Fig. 4(d)).

D. Benchmarking on a Biomedical Signal Processing Task
In Section III we will present a use-case of bio-signal processing where FPGA and memristive DNN accelerators are implemented and simulated. These are later compared to equivalent existing implementations 1 using DNN accelerators and neuromorphic processors from [39]. To perform comparisons, we use the same hand-gesture recognition task as in [39].
Hand-gesture recognition is an important task in medical settings such as prosthetic control, which can be performed using EMG biomedical signals, hand-gesture images, or a combination of both. Here, the adopted hand-gesture dataset [39] is a collection of 5 hand gestures recorded with two sensor modalities: muscle activity from a Myo armband that senses EMG electrical activity in forearm muscles, and a visual input in the form of DVS events. Moreover, the dataset provides accompanying video captured from a traditional frame-based camera, i.e., images from an Active Pixel Sensor (APS) to feed DNNs. Recordings were collected from 21 subjects including 12 males and 9 females between the ages 25 and 35, and were taken over three separate sessions.
For each implementation, we compare the mean and standard deviation of the accuracy obtained over a 3-fold cross validation, where each fold encapsulates all recordings from a given session. Additionally, for all implementations, we compare the energy and time required to perform inference on a single input, as well as the Energy-Delay Product (EDP), which is the average energy consumption multiplied by the average inference time.

III. DNN ACCELERATORS TOWARDS HEALTHCARE AND
BIOMEDICAL APPLICATIONS In this Section, we cover the use of CMOS and memristors in DL acceleration. We discuss how they use different strategies to achieve two of the key DNN acceleration goals, namely MAC parallelism and reduced memory access. We also discuss and review FPGAs as an alternative reconfigurable DNN accelerator platform, which has shown great promise in the healthcare and biomedical domains.

A. CMOS DNN accelerators
General edge-AI CMOS accelerator chips can be used for DNN-enabled healthcare IoT and PoC systems. Therefore, within this subsection, we first review a number of these chips and provide examples of potential healthcare applications they can accelerate. We will also explore some common approaches to CMOS-driven acceleration of AI algorithms using massive MAC parallelism and reduced memory access, which are useful for both edge-AI devices and offline data center scale acceleration. We then delve deeper into one of the more renowned approaches, namely, the use of systolic arrays, and show how a large accelerator developed using systolic arrays has been used to perform a breast cancer detection training and acceleration [9].
1) Edge-AI DNN accelerators suitable for biomedical applications: The research and market for ASICs, which focus on a new generation of microprocessor chips dedicated entirely to machine learning and DNNs, have rapidly expanded in recent years. Table II shows a number of these CMOSdriven chips, which are intended for portable applications. There are many other examples of AI accelerator chips (for a comprehensive survey see [44]), but here we picked several prolific examples, which are designed specifically for DL using DNNs, RNNs, or both. We have also included a few general purpose AI accelerators from Google [45], Intel [46], and Huawei [47].
Although developed for general DNNs, the accelerators shown in Table II

Inputs to neuromorphic processors are temporally encoded
Inputs to ANNs are processed in batches, which are propagated serially Time-series signals can be temporally encoded or digitally sampled  Fig. 4. DNNs and SNN neuromorphic processors adopt different operation models. In DNNs, inputs are processed in batches which propagate serially. Consequently, they require clocks for process synchronization. SNNs are asynchronous and process temporally encoded inputs independently. Time series signals, such as the EMG signal presented in (a) can be either (b) temporally encoded using spike train encoding schemes such as [37], before being fed into (j) neuromorphic processors, or (c) digitally sampled, before being concatenated into batches, to be fed into (d) DNNs. Similarly, photographs captured from (e) lenses can be (i) temporally encoded into spike trains using (h) DVSs [43] or (f) digitally encoded using conventional cameras to build (g) image frames. a few exemplar healthcare and biomedical applications that are picked based on the demonstrated capacity of these accelerators to run (or train [48]) various well-known CNN architectures such as VGG, ResNet, MobileNet, AlexNet, Inception, or RNNs such as LSTMs, or combined CNN-RNNs. It is worth noting that most of the available accelerators are intended for CNN inference, while only some [49], [50], [51] also include recurrent connections for RNNs acceleration. The Table shows that the total power per chip in most of these devices is typically in the range of hundreds of mW, with a few exceptions consuming excessive power of around 10 Watts [46], [47]. This is required to avoid large heat sinks and to satisfy portable battery constraints. The Table also shows the computing capability per unit time (column 'Computational Power (GOP/s)'). Regardless of power consumption, this column reveals the computational performance and consequently the size of a network one can compute per unit time. It is demonstrated that several of these chips can run large and deep CNNs such as VGG and ResNet, which enable them to perform complex processing tasks within a constrained edge power budget.

ANNs require clocks for process synchronization
For instance, it has been previously shown in [53] that VGG CNN (shown to be compatible with Cambricon-x [52]), can successfully analyze ECog signals. Therefore, considering the power efficiency of Cambricon-x, it can be used to implement a portable automatic ECog analyzer for PoC diagnosis of various cardiovascular diseases [66]. Similarly, Eyeriss [54] can run VGG-16, which is shown to be effective in diagnosing thyroid cancer [55]. In addition, Eyeriss can run AlexNet for several different medical imaging applications [30]. Therefore, Eyeriss can be used as a mobile diagnostic tool that can be integrated into or complement medical imaging systems at the PoC. Origami [56] is another CNN accelerator chip, which can be used for other healthcare applications based on a CNN. For instance, [57] proposes a CNN-based ECG analysis for heart monitoring, where Origami can be used to develop a smart healthcare IoT edge device. Similarly, the CNN processor proposed in [58] is shown to be able to run AlexNet, which can be deployed in a PoC ultrasound image processing system [59]. Envision [60] is another accelerator that has the capability to run large-scale CNNs. It can also be used as an edge inference engine for a multi-layer CNN for EEG/ECog feature extraction for epilepsy diagnosis [61]. Neural processor [62] is another CNN accelerator that is shown to be able to run Inception V3 CNN, which can be used for skin cancer detection [11] at the edge. LNPU [48] is the only CNN accelerator shown in Table II, which unlike the others can perform both learning  [31], -Health monitoring by heart rate variability analysis using ECG analysis by a bidirectional LSTM [34] and inference of a deep network such as AlexNet and VGG-16, for applications including on edge medical imaging [30] and cancer diagnosis [55].
Unlike the above discussed chips that are capable of running only CNNs, DNPU [49], Thinker [50], and UNPU [51] are capable of accelerating both CNNs and RNNs. This feature makes them suitable for a wider variety of edge-based biomedical applications such as ECG analysis for BCI using a cascaded RNN-CNN [32], or PoC MRI construction from motion ultrasounds using a long-term recurrent CNN [63], or intelligent medical consultation using a CNN-RNN [33]. Table II lists three general purpose AI accelerator chips, which have been deployed for low-cost and easy-to-access skin cancer detection using MobileNet V1 CNN [24], on edge health monitoring for fall detection using LSTMs [64], chest x-ray analysis using ResNet CNN [65], cardiovascular arrhythmia detection from ECG using an LSTM [31], or heart rate variability analysis from ECG signals through a bidirectional LSTM [34], just to name a few.
2) Common approaches to CMOS-driven DL acceleration: Accelerators will typically target either data center use or embedded 'edge-AI' acceleration. Edge chips, such as those discussed above, must operate under restrictive power budgets (e.g., within thermal limits of 5 W) to cope with portable battery constraints. While the scale of tasks, input dimension capacity, and clock speeds will differ between edge-AI and modular data center racks, both will adopt similar principles in the tasks they seek to optimize.
Sequential and combinational logic research is largely matured, so outside of emerging memory technologies, the dominant hardware benefits are brought on by optimizing data flow and architecture. An early example is the neuFlow system-onchip (SoC) processor which relies on a grid of processing tiles, each made up of a bank of processing operators and a multiplexer based on-chip router [67]. The processing operator can serially perform primitive computation (MUL, DIV, ADD, SUB, MAX), or a parallelized 1D/2D convolution. The router configures data movement between tiles to support streaming data flow graphs.
Since the development of neuFlow, over 100 startups and companies have developed, or are developing, machine learning accelerators. The Neural Processing Unit (NPU) [68] generalizes the work from neuFlow by employing eight processing engines which each compute a neuron response: multiplication, accumulation, and activation. If a program could be partitioned such that a segment of it can be calculated using MACs, then it would be partially computed on the NPU. This made it possible to go beyond MLP neural networks. The NPU was demonstrated to perform Sobel edge detection and fast Fourier transforms as well.
NVIDIA coupled their expertise in developing GPUs with machine learning dedicated cores, namely, tensor cores, which are aimed at demonstrating superior performance over regular Compute Unified Device Architecture (CUDA) cores [16]. Tensor cores target mixed-precision computing, with their NVIDIA Tesla V100 GPU combining 672 tensor cores on a single unit. By merging the parallelism of GPUs with the application specific nature of tensor cores, their GPUs are capable of energy efficient general compute workloads, as well as 12 trillion floating-point operations per seconds (TFLOPSs) of matrix arithmetic.
Although plenty of other notable architectures exist (see Table II), a pattern begins to emerge, as most specialized processors rely on a series of sub-processing elements which each contribute to increasing throughput of a larger processor. Whilst there are plenty of ways to achieve MAC parallelism, one of the most renowned techniques is the systolic array, and is utilized by Groq [69] and Google, amongst numerous other chip developers. This is not a new concept: systolic architectures were first proposed back in the late 1970s [70], [71], and have become widely popularized since powering the hardware DeepMind used for the AlphaGo system to defeat Lee Sedol, the world champion of the board game Go in October 2015. Google also uses systolic arrays to accelerate MACs in their TPUs, just one of many CMOS ASICs used in DNN processing [13]. Here, we explain what systolic arrays are and how they can be used to decrease memory access frequency and increase MAC parallelism, towards efficient ANN accelerators.
3) Systolic arrays for DNN acceleration: In general purpose computing, there is no knowing what the next instruction could be. The result of every operation must be stored in memory, while awaiting further instructions from the processor.
Energy is consumed in reading from memory, in writing to memory, and time is wasted by shuttling information on a limited bandwidth bus to and from the processor (Fig. 5(a)).
On the other hand, neural networks are deterministic. Once the network has been trained, every operation that the input data is subject to has already been pre-determined. This allows a single element of information, such as one pixel of an image, to have many operations applied to it prior to being stored in memory. Systolic arrays loosely draw inspiration from the cardiovascular system, where blood is pumped through various subsystems prior to returning to the heart. Similarly, in systolic processing, data flows through many processing elements before it returns to memory (Fig. 5(b)). In fact, the word systolic is derived from the cardiac cycle.
The appeal of systolic arrays is that they can come in many forms, designed for different tasks using repeatable and modular blocks. As a simple case study of how systolic arrays parallelize operations with infrequent memory write cycles, we can consider a 3x3 matrix multiplication by referring to Fig. 6. Here, the processing element is designed to multiply two inputs together, and accumulate it with all future products. The input data is a matrix of values x m,n and a weight matrix of values w m,n . Multiplying these two matrices together is an efficient way to compute a sum-of-products, or a MAC operation.
As depicted in Fig. 6, input data is carefully orchestrated in time such that it naturally flows in rhythm with incoming weight data. At T=1, x 0,0 is multiplied with w 0,0 . At T=2, x 0,0 flows to the right and is multiplied by the next weight in sequence, w 0,1 . The weight w 0,0 flows down and is multiplied by the next input, x 1,0 . Another input-weight pair enters the array: x 0,1 and w 1,0 are multiplied together in the topleft processing element, and summed with the result of the previous time-step.
This process is repeated, until all inputs have traversed to the right of the array, and all weights have traversed to the bottom of the array, giving the result shown at T=7. This is equivalently performing matrix multiplication, which is the dominant operation in a DNN. Every element of the matrix can be computed in this way, without having to store any intermediate results in main memory. 4) CMOS-based systolic arrays used in biomedical applications: Googles TPU utilizes a 128×128 systolic array, which enables 180 TFLOPSs, while v3 reaches up to 100 peta-FLOPS (PFLOPS). The modularity of systolic arrays makes them easily scalable for a large number of interconnected TPUs, a necessary feature for data center use. Even with a relatively slow clock (e.g., 700 MHz for TPU v1), systolic arrays are highly parallel meaning there are numerous matrices being processed simultaneously. TPUs were used in the seminal work from Ref. [9] where an ensemble of three DNNs were used to surpass radiologist performance in breast cancer detection. This network was trained on a set of over 100,000 images, many of which were at 4K resolution and required the development of substantial infrastructure to make training such a system possible. The results demonstrated significant reduction in both false positives and false negatives. Notably, the system was able to generalize from being trained on UK-  based data sets to competitive performance on USA-based images.
Overall, systolic arrays make efficient use of a limited memory bandwidth. While the connection from processor to memory is a bottleneck, the interconnections between processing elements can be very fast. The drawback is that if the required computation cannot be mapped into the processing elements functions, such as a MAC, then it cannot be implemented.

B. Memristive DNNs
To achieve the two aforementioned key DNN acceleration goals, i.e. massive MAC parallelism and reduced memory access, many studies have leveraged memristors [72], [73], [74], [75] as weight elements in their DNN and SNN [76], [77] architectures. Memristors are often referred to as the fourth fundamental circuit element, and can adapt their resistance (conductance) to changes in the applied current or voltage. This is similar to the adaptation of neural synapses to their surrounding activity while learning. This adaptation feature is integral to the brain's in-memory processing ability, which is missing in today's general purpose computers. This in-situ processing can be utilized to perform parallel MAC operations inside memory, hence, significantly improving DNN learning and inference. This is achieved by developing memristive crossbar neuromorphic architectures, which are projected to achieve approximately 2500-fold reduction in power and a 25-fold increase in acceleration, compared to state-of-the-art specialized hardware such as GPUs [72].
1) Memristive crossbars for parallel MAC and VMM operations: A memristive crossbar that can be fabricated using a variety of device technologies [77], [78] can perform analog MAC operations in a single time-step (see Fig. 7(a)). This reduces the time complexity to its minimum (O(1)), and is achieved by carrying out multiplication at the place of memory, in a non-Von Neumann structure. Using this wellknown approach, VMM can be parallelized as demonstrated in Fig. 7(b), where the vector of size M values represented Multiply-accumulate operation  2) Mapping memristive crossbars to DNN layers: Although implementing fully-connected DNN layers is straightforward by mapping the weights to crossbar point memristors and having the inputs represented by input voltages, implementing a complex CNN requires mapping techniques to convert convolution operations to MAC operations. A popular approach to perform this conversion is to use an unrolling (unfolding) operation that transforms the convolution of input feature maps and convolutional filters to MAC operations. We have developed a software platform named MemTorch [79] that will be introduced in subsequent sections, to perform this mapping as well as a number of other operations, for converting DNNs to Memristive DNNs (MDNNs). The mapping process implemented in MemTorch is illustrated in the left panel in Fig. 8. The figure shows that the normal input feature maps and convolution filters (shown in gray shaded area) are unfolded and reshaped (shown in the cyan shaded area) to be compatible to memristive crossbar parallel VMM operation. It is worth noting that, the convolutional filters that can be applied to the input feature maps have a direct relationship with the required crossbar sizes. Furthermore, the resulting hardware size or required time, depends on the size of the input feature maps [80].
3) Peripheral circuitry for memristive DNNs: In addition to the memristive devices that are used as programmable elements in MDNN architectures, various peripheral circuitry is required to perform feed-forward error-backpropagation learning in MDNNs [74]. This extra circuitry may include: (i) a conversion circuit to translate the input feature maps to input voltages, which for programming memristive devices are usually Pulse Width Modulator (PWM) circuits, (ii) current integrators or sense amplifiers, which pass the current read from every column of the memristive crossbar to (iii) analog to digital converters (ADCs), which pass the converted voltage to (iv) an activation function circuit, for forward propagation, and for backward error propagation (v) the activation function derivative circuit. Other circuits required in the error backpropagation path include (vi) backpropagation values to PWM voltage generators, (vii) backpropagation current integrators, and (viii) backpropagation path ADCs. In addition, an update module that updates network weights based on an algorithm such as SGD is required, which is usually implemented in software. After the update, the new weight values should be written to the memristive crossbar, which itself requires Bit-Line (BL) and Word-line (WL) switch matrices to address the memristors for update, as well as a circuit to update the memristive weights. There are different approaches to implement this circuit such as that proposed in [81], while others may use software ex-situ training where the new weight values are calculated in software and transferred to the physical memristors through peripheral circuitry [74].

4) Memristive device nonidealities:
Although ideal memristive crossbars have been projected to remarkably accelerate DNN learning and inference and drastically reduce their power consumption [72], [73], device imperfections observed in experimentally fabricated memristors impose significant performance degradation when the crossbar sizes are scaled up for deployment in real-world DNN architectures, such as those required for healthcare and biomedical applications discussed in subsection III-A. These imperfections include nonlinear asymmetric and stochastic conductance (weight) update, device temporal and spatial variations, device yield, as well as limited on/off ratios [72]. To minimize the impact of these imperfections, specific peripheral circuitry and system-level mitigation techniques have been used [82]. However, these techniques add significant computation time and complexity to the system. It is, therefore, essential to take the effect of these nonidealities into consideration before utilizing memristive DNNs for any healthcare and medical applications, where accuracy is critical. In addition, there is a need for a unified tool that reliably simulates the conversion of a pre-trained DNN to a MDNN, while critically considering experimentally modeled device imperfections [79].

5) Conversion of DNN to MDNN while considering memristor nonidealities:
Due to the significant time and energy required to train new large versions of DNNs for challenging cognitive tasks, such as biomedical and healthcare data processing [9], [83], the training of the algorithms is usually undertaken in data centers [9], [13]. The pretrained DNN can then be transferred to be used on memristive crossbars. There exist several different frameworks and tools that can be used to simulate and facilitate this transition [84]. In a recent study, we have developed a comprehensive tool named MemTorch, which is an open source, general, high-level simulation platform that can fully integrate any behavioral or experimental memristive device model into crossbar architectures to design MDNNs [79].
Here, we utilize the benchmark biomedical signal processing task explained in subsection II-D to demonstrate how pretrained DNNs can be converted to equivalent MDNNs, and how non-ideal memristive devices can be simulated within MDNNs prior to hardware realization. The conversion process, which can be generalized to other biomedical models using MemTorch, is depicted in Fig. 8.
The targeted MDNNs are constructed by converting linear and convolutional layers from PyTorch pre-trained DNNs to memristive equivalent layers employing 1-Transistor-1-Resistor (1T1R) crossbars. A double-column scheme is used to represent network weights within memristive crossbars. The converted MDNN models are tuned using linear regression, as described in [79]. The complete and detailed process and the source code of the network conversion for the experiments shown in this subsection are provided in a publicly accessible complementary Jupyter Notebook 2 .
During the conversion, any memristor model can be used. For the benchmark task, a reference VTEAM model [85] is instantiated using parameters from Pt/Hf/Ti Resistive Random Access Memory (RRAM) devices [86], to model all memristive devices within converted linear and convolutional layers. As already mentioned, memristive devices have inevitable variability, which should be taken into account when implementing an MDNNs for learning and/or inference. Also, depicted in Fig. 8 are visualizations of two non-ideal device characteristics: the finite number of conductance states and device-to-device variability. Using MemTorch [79], not only can we convert any DNNs to an equivalent MDNNs utilizing any memristive device model, we are also able to comprehensively investigate the effect of various device non-idealities and variation on the performance of a possible MDNN, before it is physically realized in hardware.
In order to demonstrate an example which includes variability in our MDNN simulations, device-device variability is introduced by sampling R OFF for each device from a normal distribution withR OFF = 2k5Ω with standard deviation 2σ, and R ON for each device from a normal distribution withR ON = 100Ω with standard deviation σ.
In Fig. 9, for the converted memristive MLP and CNN that process APS hand-gesture inputs, we gradually increase σ from 0 to 500, and compare the mean test set accuracy across the three folds. As can be observed from Fig. 9, with increasing device-to-device variability, i.e. the variability of R ON and R OFF , the performance degradation increases across all networks. For all simulations, R ON and R OFF are bounded to be positive.

6) Memristive DNNs towards biomedical applications:
Although some previous small-scale MDNNs have been simulated for biomedical tasks such as cardiac arrhythmia classification [87], or have been implemented on a physical programmable memristive array for breast cancer diagnosis [88], there currently exists no significant MDNN, even at simulation-level, which has realized a large-scale biomedical processing task.
Similar to the recent advances in CMOS-driven DNN accelerator chips discussed in subsection III-A, there have been promises in partial [73] or full [74] realizations of MDNNs in hardware, which are shown to achieve significant energy saving compared to state-of-the-art GPUs. However, unlike their CMOS counterparts, these implementations have been only able to perform simple tasks such as MNIST and CIFAR classification. This is, of course, not suitable for implementing large-scale CNNs and RNNs, which as shown in subsection III-A are required for biomedical and healthcare tasks dealing with image [30] or temporal [31] data types.
In addition, following similar optimization strategies as those used in CMOS accelerators, [89] has investigated, in simulations, the use of quantized and binarized MDNNs and their error tolerance in a biomedical ECG processing task and has shown their potential to achieve significant energy savings compared to full-precision MDNNs. However, due to the many intricacies in the design process and considering the aforementioned peripheral circuitry that may offset the benefits gained by using MDNNs, full hardware design is required before the actual energy saving of such binarized MDNNs can be verified.

C. FPGA DNNs
FPGAs are fairly low-cost reconfigurable hardware that can be used in almost any hardware prototyping and implementation task, significantly shortening the time-to-market of an electronic product. They also provide parallel computation, which is essential when simultaneous data processing is required such as processing multiple ECG channels in parallel. Furthermore, there exists a variety of High Level Synthesis (HLS) tools and techniques [90], [91] that facilitate FPGA prototyping without the need to directly develop timeconsuming low-level Hardware Description Language (HDL) codes [92]. These tools allow engineers to describe their targeted hardware in high-level programming languages such as C to synthesize them to Register Transfer Level (RTL). The tools then offload the computational-critical RTL to run as kernels on parallel processing platforms such as FPGAs [93].
Here, we demonstrate an exemplar process of accelerating DNNs used for the benchmark biomedical signal processing task explained in subsection II-D. For our acceleration, we use fixed-point parameter representations on a Starter Platform for OpenVINO Toolkit FPGA using OpenCL. OpenCL [90] is an HLS framework for writing programs that execute across heterogeneous platforms. OpenCL specifies programming languages (based on C99 and C++11) for programming the compute devices and Application Programming Interfaces (APIs) to control and execute its developed kernels on the devices, where depending on the available computation resources, an accelerator can pipeline and execute all work items in parallel or sequentially. Fig. 10 depicts the compilation flow we adopted. The trained DNN PyTorch model is first converted to .prototxt and .caffemodel files using Caffe. All weights and biases are then converted to a fixed point representation using MAT-LAB's Fixed-point toolbox using word length and fractional bit lengths defined in [98], prior to being exported as a single binary .dat file for integration with PipeCNN, which is used to generate the necessary RTL libraries, and to perform compilation of the host executable and the FPGA bit-stream. We used Intel's FPGA SDK for OpenCL 19.1, and provide all files used during the compilation shown in Fig. 10 in a publicly accessible complementary GitHub repository 3 .
2) FPGA-based DNNs for biomedical applications: Despite the many FPGA-based DNN accelerators available [91], only a few have been developed specifically for biomedical applications such as ECG anomaly detection [99], or real-time mass-spectrometry data analysis for cancer detection [100], where the authors show that application-specific parameter quantization and customized network design can result in significant inference speed-up compared to both CPU and GPU. In addition, the authors in [101] have developed an FPGA-based BCI, in which a MLP is used for reconstructing ECog signals. In [102], the authors have implemented an EEG processing and neurofeedback prototype on a low-power but low-cost FPGA and then scaled it on a high-end Ultra-scale Virtex-VU9P, which has achieved 215 and 8 times power efficiency compared to CPU and GPU, respectively. For the EEG processing, they developed an LSTM inference engine.
It is projected that, by leveraging specific algorithmic design and hardware-software co-design techniques, FPGAs can provide >10 times energy-delay efficiency compared to stateof-the-are GPUs for accelerating DL [91]. This is significant for realizing portable and reliable healthcare applications. However, FPGA design is not as straightforward as high-level designs conducted for DL accelerators and requires skilled engineers and stronger tools, such as those offered by the GPU manufacturers.
In the next section, we provide our analysis and perspective on the use of the three hardware technologies discussed in this section for DL-based biomedical and healthcare applications. We also discuss how SNN-based neuromorphic processors can benefit edge-processing for biomedical applications.

IV. ANALYSIS AND PERSPECTIVE
The use of ANNs trained with the backpropagation learning algorithm in the domain of healthcare and for biomedical applications such as cancer diagnosis [108] or ECG monitoring [109] dates back to the early 90s. These networks, were typically small-scale networks run on normal workstations. As they were not deep and did not have too many parameters, they did not demand high-performance accelerators. However, with the resurgence of CNNs in the early 2010s followed by the rapid spread of DNNs and large data-sets, came the need for high-speed specialized processors. This need resulted in repurposing GPUs and actively researching other hardware and design technologies including ASIC CMOS chips (see Table II) and platforms [13], memristive crossbars and in-memory computing [73], [74], [80], and FPGA-based designs for DNN training [96], [97] and inference [94]. Despite notable progress in deploying non-GPU platforms for DL acceleration, similar to other data processing tasks, biomedical and healthcare tasks have mainly relied on standard technologies and GPUs. Currently, depending on the size of the required DNN, its number of parameters, as well as the available training dataset size, biomedical DL tasks are usually "trained" on high-performance workstations with one or more GPUs [12], [18], on customized proprietary processors such as Google TPU [9], or on various Infrastructure-asa-Service (IaaS) provider platforms, including Nvidia GPU cloud, Google Cloud, and Amazon Web Services, among others. This is mostly due to (i) the convenience these platforms provide using high-level languages such as Python; (ii) the availability of wide-spread and open-source DL libraries such as TensorFlow and PyTorch; and (iii) strong community and/or provider support in utilizing GPUs and IaaS for training various DNN algorithms and applications.
However, "inference" can benefit from further research and development on emerging and mature hardware and design technologies such as those discussed in this paper, to open up new opportunities for deploying healthcare devices closer to the edge, paving the way for low-power and low-cost DL accelerators for PoC devices and healthcare IoT. Despite this fact, hardware implementations of biomedical and healthcare inference engines are very sparse. Table III lists a summary of the available hardware implementations and hardware-based simulations of DNNs used for healthcare and biomedical signal processing applications, using the three hardware technologies covered herein. In addition, the table shows existing biomedical signal processing tasks implemented on generic low-power spiking neuromorphic processors.

A. CMOS technology has been the main player for DL inference in the biomedical domain
Similarly to general-purpose GPUs that are CMOS-based, all the other current non-GPU DL inference engines are implemented in CMOS. Therefore, it is obvious that most of the future edge-based biomedical platforms would rely on these inference platforms. In Table II, we listed a number of these accelerators that are mainly developed for low-power mobile applications. We also mentioned a set of potential healthcare and biomedical tasks that can be realized using them. However, before the deployment of any edge-based DL accelerators for biomedical and healthcare tasks, some challenges need to be overcome. A non-exhaustive list of these obstacles include: (i) the power and resource constraints of available mobile platforms, which despite significant improvements are still not suitable for complex medical tasks; (ii) the need to verify that a DL system can generalize beyond the distribution they are trained and tested on; (iii) bias that is inherent to datasets which may have adverse impacts on classification across different populations; (iv) confusion surrounding the Image-based breast cancer diagnosis [9] Ensemble of CNNs CMOS (Google TPU) Energy-efficient multi-class ECG classification [38] Spiking RNN CMOS EMG signal processing [39] Spiking CNN/MLP CMOS ECG signal processing [103] Spiking RNN CMOS EMG signal processing [104] Spiking RNN CMOS EMG signal processing [105] Feed-forward SNN CMOS EMG and EEG signal processing [106] Recurrent 3D SNN CMOS EEG and LFP signal processing [107] TrueNorth-compatible CNN CMOS ECG processing for cardiac arrhythmia classification [87] MLP Memristors † Breast cancer diagnosis [88] MLP Programmable Memristor-CMOS system ECG signal processing [89] Binarized CNN Memristors † ECG arrhythmia detection for hearth monitoring [99] MLP FPGA Mass-spectrometry for real-time cancer detection [100] MLP FPGA ECog signal processing for BCI [101] MLP FPGA EEG processing for energy-efficient Neurofeedback devices [102] LSTM FPGA liability of AI algorithms in high-risk environments [110]; and (v) the lack of a streamlined workflow between medical practitioners and DL. While the latter challenges are matters of legality and policy, the former issues highlight the fundamental need to understand where dataset bias comes from, and how to improve our understanding of why neural networks learn the features they do, such that they may generalize across populations in a manner that is safe for receivers of medical care.
In addition, to make the use of any accelerators possible for general as well as more complex biomedical applications, the field requires strong hardware-software co-design to build up hardware that can be readily programmed for biomedical tasks. One successful example of a solid hardware-software co-design for a DL-customized CMOS platform (shown in Table III) is the Google TPU [13], which while generic, has been used along with complex tailored software for humansurpassing medical imaging tasks [9]. Google has used a similar CMOS TPU technology to design inference engines [45], which are very promising as edge hardware to enable mobile healthcare care applications. The main reason for this promise is the availability of the solid software platforms (such as TensorFlow Light) and the community support for the Google TPU.
Overall, great advancements have happened for DL accelerators in the past several years and they are currently stemming in various aspects of our life from self-driving cars to smart personal assistants. After overcoming a number of obstacles such as those mentioned above, we may be also able to widely integrate these DL accelerators in healthcare and biomedical applications. However, for some medical applications such as monitoring that requires always-on processing, we still need systems with orders of magnitude better power efficiency, so they can run on a simple button battery for a long time. To achieve such systems, one possible approach is to process data only when available and make our processing asynchronous. A promising method to achieve such goals is the use of braininspired SNN-based neuromorphic processors.

B. Towards edge processing for biomedical applications with neuromorphic processors
Although most of the efforts presented in this work focused on DNN accelerators, there are also notable efforts in the domain of SNN processors that offer complementary advantages, such as the potential to reduce the power consumption by multiple orders of magnitude, and to process the data in real time. These so-called neuromorphic processors are ideal for end-to-end processing scenarios for example in wearable devices, where the streaming input needs to be monitored in continuous time in an always-on manner.
There are already some works in the direction of processing biomedical signals that explore both mixed analog-digital and digital neuromorphic platforms, showing promising results for always-on embedded biomedical systems. Table IV shows a summary of today's large scale neuromorphic processors, used for biomedical signal processing. The first chip presented in this table is DYNAP-SE [111], a multi-core mixed-signal neuromorphic implementation with analog neural dynamics circuits and event-based asynchronous routing and communication circuits. The DYNAP-SE chip has been used to implement four of the seven SNN processing systems listed in Table III. These SNNs are used for the classification or detection of EMG [104], [105] and ECG [103], [38]. The DYNAP-SE was also used to build a spiking perceptron as part of a design to classify and detect High-Frequency Oscillations (HFO) in human intracranial EEG [42].
In [38], [103], [104] a spiking RNN is used to integrate the ECG/EMG patterns temporally and separate them in a linear fashion to be classifiable with a linear read-out. Support Vector Machine (SVM) and linear least square approximation is used in the read out layer for [103], [38] and overall accuracy of 91% and 95% for anomaly detection were reached respectively. In [104], the timing and dynamic features of the spiking RNN on EMG recordings was investigated for classifying different hand gestures. In [105] the performance of a feedforward SNN and a hardware-friendly spiking learning algorithm for hand gesture recognition using superficial Biosignal processing application EMG [105], ECG [103], HFO [42] EMG and EEG [106] EEG and LFP [107] EMG [39] EMG [39] EMG was investigated and compared to traditional machine learning approaches, such as SVM. Results show that applying SVM on the spiking output of the hidden layer achieved a classification rate of 84%, and the spiking learning method achieved 74% with a power consumption of about 0.05 mW . The consumption was compared to state-of-the-art embedded system showing that the proposed spiking network is two orders of magnitude more power efficient [112], [113].
The other neuromorphic platforms listed in Table IV include digital architectures such as SpiNNaker [114], TrueNorth [115] and Loihi [116]. SpiNNaker has been used for EMG and EEG processing and obtained results show a better classification accuracy compared to traditional machine learning methods [106]. In [107], the authors developed a framework for decoding EEG and LFP using CNNs. The network was first developed in Caffe and the result was then used as a basis for building a TrueNorth-compatible neural network. The TrueNorth-compatible network achieved the highest classification, around 76%. Recently, the benchmark hand-gesture classification introduced in subsection II-D, was processed and compared on two other digital neuromorphic platforms, i.e. Loihi and ODIN/MorphIC [117], [118]. A spiking CNN was implemented on Loihi and a spiking MLP was implemented on ODIN/MorphIC [39]. The results achieved using these networks are presented in Table V.
On-chip adaptation and learning mechanisms, such as those present in some of the neuromorphic devics listed in Table IV, could be a game changer for personalized medicine, where the system can adapt to each patient's unique bio signature and/or drift over time. However, the challenge of implementing efficient on-chip online learning in these types of neuromorphic architectures has not yet been solved. This challenge lies on two main factors: locality of the weight update and weight storage.
Locality: There is a hardware constraint that the learning information for updating the weights of any on-chip network should be locally available to the synapse, otherwise most of the silicon area would be taken by the wires, required to route the update information to it. As Hebbian learning satisfies this requirement, most of the available on-chip learning algorithms focus on its implementation in forms of unsupervised/semisupervised learning [117], [119]. However, local Hebbianbased algorithms are limited in learning static patterns or using very shallow networks [120]. There are also some efforts in the direction of on-chip gradient-descent based methods which implement on-chip error-based learning algorithms where the least mean square of a neural network cost function is minimized. For example, spike-based delta rule is the most common weight update used for single-layer networks which is the base of the back-propagation algorithm used in the vast majority of current multi-layer neural networks. Single layer mixed-signal neuromorphic circuit implementation of the delta rule have already been designed [121] and employed for EMG classification [105]. Expanding this to multi-layer networks involves non-local weight updates which limits its on-chip implementation. Making the backpropagation algorithm local is a topic of on-going research [122], [123], [124].
Weight storage: The holy grail weight storage for online on-chip learning is a memory with non-volatile properties whose state can change linearly in an analog fashion. Nonvolatile memristive devices provide a great potential for this. Therefore, there is a large body of literature in combining the maturity of CMOS technology with the potential of the emerging memories to take the best out of the two worlds.
The integration of CMOS technology with that of the emerging devices has been demonstrated for non-volatile filamentary switches [125] already at a commercial level [126]. There have also been some efforts in combining CMOS and memristor technologies to design supervised local error-based learning circuits using only one network layer by exploiting the properties of memristive devices [121], [127], [128].
Apart from the above-mentioned benefits in utilizing memristive devices for online learning in SNN-based neuromorphic chips, as discussed in subsection III-B, memristive devices have also shown interesting features to improve the power consumption and delay of conventional DNNs. However, as shown in Table III, memristor-based DNNs are very sparse in the biomedical domain, and existing works are largely based only on simulation.
C. Why is the use of MDNNs very limited in the biomedical domain?
Currently there are very few hardware implementations of biomedical MDNNs that make use of general programmable memristive-CMOS, and only one programmed to construct an MLP for cancer diagnosis. We could also find two other memristive designs in literature for biomedical applications (shown in Table III), but they are only simulations considering memristive crossbars. This sparsity is despite the significant advantages that memristors provide in MAC parallelization and in-memory computing paradigm, while being compatible with CMOS technology. These features make memristors ideal candidates for DL accelerators in general, and for portable and edge-based healthcare applications in particular, because they have stringent device size and power consumption requirements. To be able to use memristive devices in biomedical domain, though, several of their shortcomings such as limited endurance, mismatch, and analog noise accumulation must be overcome first. This demands further research in the materials, as well as the circuit and system design side of this emerging technology, while at the same time developing facilitator opensource software [79] to support MDNNs. Furthermore, investigating the same techniques utilized in developing CMOSbased DL accelerators such as limited precision data representation [80], [89] and approximate computing schemes can lead to advances in developing MDNNs and facilitate their use in biomedical domains.
D. Why and when to use FPGA for biomedical DNNs? Table III shows that FPGA is a fairly popular hardware technology for implementing simple DL networks such as MLPs and in one case, a complex LSTM. The table also shows that FPGAs are mainly used for signal processing tasks and have not been widely used to run complex DL architectures such as CNNs. This is mainly because they have limited on-chip memory and low bandwidth compared to GPUs. However, they present notable benefits in terms of significantly shorter development time compared to ASICs, and much lower power consumption than typical GPUs. Besides, significant power and latency improvement can be gained by customizing the implementation of various components of a DNN on an FPGA, compared to running it on a general-purpose CPU or GPU [100], [102]. For instance, in [102], EEG signals are processed on FPGAs using two customized hardware blocks for (i) parallelizing MAC operations and (ii) efficient recurrent state updates, both of which are key elements of LSTMs. This has resulted in almost an order of magnitude power efficiency compared to GPUs. This efficiency is critical in many edgecomputing applications including DNN-based point-of-care biomedical devices [20] and healthcare IoT [19], [57].
Another benefit of FPGAs is that a customized efficient FPGA design can be directly synthesized into an ASIC using a nanometer-node CMOS technology to achieve even more benefits. For instance, [102] has shown near 100 times energy efficiency improvement as an ASIC in a 15-nm CMOS technology, compared to its FPGA counterpart.
Although low-power consumption and affordable cost are two key factors for almost any edge-computing or near-sensor device, these are even more important for biomedical devices such as wearables, health-monitoring systems, and PoC devices. Therefore, FPGAs present an appealing solution, where their limitations can be addressed for a customized DNN using specific design methods such as approximate computing [97] and limited-precision data [94], [96], depending on the cost, required power consumption, and the acceptable accuracy of the biomedical device.

E. Benchmarking EMG processing across multiple DNN and SNN hardware platforms
In Table V, we compare our FPGA and memristive implementations to other DNN accelerators and neuromorphic processors from [39]. Input and hidden layers are sequenced with the ReLU activation function, and output layers are fed through Softmax activation functions to determine class probabilities. Dropout layers are used in all networks to avoid over-fitting. The DNN architectures are determined in the table caption. Further implementation details can be found in [39].
The platforms used for each system in Table V are as follows: ODIN+MorphIC [117], [118] and Loihi [116] neuromorphic platforms were used for spiking implementations; NVIDIA Jetson Nano was used for all embedded GPU implementations; OpenVINO Toolkit FPGA was used for all FPGA implementations, and MemTorch [79] was used for converting the MLP and CNN networks to their corresponding MDNNs to determine the test set accuracies of all memristive implementations.
From Table V, it can be observed that, when transitioning from generalized architectures to application specific processors, more optimized processing of a subset of given tasks can be achieved. Moving up the specificity hierarchy from GPU to FPGA to memristive networks shows orders of magnitude of improvement in both MLP and CNN processing, but naturally at the expense of a generalizable range of tasks. While GPUs are relatively efficient at training networks (compared to CPUs), the impressive metrics presented by memristor (RRAM in this simulations) is coupled with limited endurance. This is not an issue for read-only tasks, as is the case with inference, but training is thwarted by the thousands of epochs of weight updates which limits broad use of RRAMs in training. Rather, more exploration in alternative resistive-based technologies such as Magnetoresistive Random Access Memory (MRAM) could prove beneficial for tasks that demand high endurance.
After determining the test set accuracy of each MDNN using MemTorch [79], we determined the energy required to perform inference on a single input, the inference time, and the Energy-Delay Product (EDP) using a similar approach to [129], for a tiled memristor architecture. All presumptions made in our calculations are listed below. Parameters are adopted from those given in a 1T1R 65nm technology, where the maximum current during inference is 3µA per cell with a read voltage of 0.3V. Each cell is capable of storing 8 bits with a resistance ratio of 100, and mapping signed weights is achieved using a dual column representation. All convolutions are performed by unrolling the kernels and performing MVMs, and the fully connected layers have the fan-in weights for a single neuron assigned to one column. Each crossbar has an aspect ratio of 256×64 to enable more analog operations per ADC when compared to a 128×128 array. Where there is insufficient space to map weights to a single array, they are distributed across multiple arrays, with their results to be added digitally. Throughput can be improved at the expense of additional arrays for convolutional layers, by duplicating kernels such that multiple inputs can be processed in parallel. The number of tiles used for each network is assumed to be the exact number required to balance the processing time of each layer. The power consumption of each current-mode 8-bit ADC is estimated to be 2×10 −4 W with an operating frequency of 40 MHz (5 MHz for bit-serial operation). The ADC latency is presumed to dominate digital addition of partial products from various tiles. The dynamic range of each ADC has been adapted to the maximum possible range for each column, and each ADC occupies a pair of columns.
The above presumptions lead to pre-silicon results that are extremely promising for memristor arrays, as shown in Table V. But it should be clear that these calculations were performed for network-specific architectures, rather than a more general application-specific use-case. That is, we assume the chip has been designed for a given neural network model. The other comparison benchmarks are far more generalizable, in that they are suited to not only handle most network topologies, but are also well-suited for training. The substantial improvement of inference time over other methods is a result of duplicate weights being mapped to enable higher parallelism, which is tolerable for small architectures, but lends to prohibitively large ADC power consumption for computer vision tasks which rely on deep networks and millions of parameters, such as VGG-16. The use of memristors as synapses in spike-based implementations may be more appropriate, so as to reduce the ADC overhead by replacing multi-bit ADCs with current sense amplifiers instead, and reducing the reliance on analog current summation along resistive and capacitive bit-lines.
Spike-based hardware show approximately two orders of magnitude improvement in the EDP from Table V when compared to their GPU and FPGA counterparts, which highlights the prospective use of such architectures in always-on monitoring. This is necessary for enhancing the prospect of ambient-assisted living, which would allow medical resources to be freed up for tasks that are not suited for automation. In general, one would expect that data should be processed in its naturalized form. For example, 2D CNNs do not discard the spatial relations between pixels in an image. Graph networks are optimized for connectionist data, such as the structure of proteins. By extension, the discrete events generated by electrical impulses such as in EMGs, EEGs and ECGs may also be optimized for SNNs. Of course, this discounts any subthreshold firing patterns of measured neuron populations. But one possible explanation for the suitability of spiking hardware for biological processes stems from the natural timing of neuronal action potentials. Individual neurons will typically not fire in excess of 100 Hz, and the average heart rate (and correspondingly, ECG spiking rate) will not exceed 3 Hz. There is a clear mismatch between the clock rate of nonspiking neural network hardware, which tend to at least be in the MHz range, and spike-driven processes. This introduces a significant amount of wastage in processing data when there is no new information to process (e.g., in between heartbeats, action potentials, or neural activity).
Nonetheless, it is clear that accuracy is compromised when relying on EMG signals alone, based on the approximately 10% decrease of classification accuracy on the Loihi chip and ODIN+MorphIC, as against their GPU/FPGA counterparts. This could be a result of spike-based training algorithms lagging behind in maturity compared to conventional neural network methods, or it could be an indication that critical information is being discarded when neglecting the subthreshold signals generated by populations of neurons. But when EMG and DVS data are combined, this multi-sensory data fusion of spiking signals positively reinforce upon each other with an approximately 4% accuracy improvement, whereas combining non-spiking, mismatched data representations leads to marginal improvements, and even a destructive effect (e.g., non-spiking CNN implementation on FPGA and memristive arrays). This may be a result of EMG and APS data taking on completely different structures. This is a possible indication that feature extraction from merging the same structural form of data (i.e., as spikes) proves to be more beneficial than combining a pair of networks with two completely different modes of data (i.e., EMG signals with pixel-driven images). This allows us to draw an important hypothesis: neural networks can benefit from a consistent representation of data generated by various sensory mechanisms. This is supported by biology, where all biological interpretations are typically represented by graded or spiking action potentials.

V. CONCLUSION
The use of DL in biomedical signal processing and healthcare promises significant utility for medical practitioners and their patients. DNNs can be used to improve the quality of life for chronically ill patients by enabling ambient monitoring for abnormalities, and correspondingly can reduce the burden on medical resources. Proper use can lead to reduced workloads for medical practitioners who may divert their attention to time-critical tasks that require a standard beyond what neural networks can achieve at this point in time.
We have stepped through the use of various DL accelerators on a disparate range of medical tasks, and shown how SNNs may complement DNNs where hardware efficiency is the primary bottleneck for widespread integration. We have provided a balanced view to how memristors may lead to optimal hardware processing of both DNNs and SNNs, and have highlighted the challenges that must be overcome before they can be adopted at a large-scale. While the focus of this tutorial and review is on hardware implementation of various DL algorithms, the reader should be mindful that progress in hardware is a necessary, but insufficient, condition for successful integration of medical-AI.
Adopting medical-AI tools is clearly a challenge that demands the collaborative attention of healthcare providers, hard-ware and software engineers, data scientists, policy-makers, cognitive neuroscientists, device engineers and materials scientists, amongst other specializations. A unified approach to developing better hardware can have pervasive impacts upon the healthcare industry, and realize significant payoff by improving the accessibility and outcomes of healthcare.
ACKNOWLEDGMENT This paper is supported in part by the European Unions Horizon 2020 ERC project NeuroAgents (Grant No. 724295).