{'fraction_non_alphanumeric': 0.03982138369642961, 'fraction_numerical': 0.010109992074384799, 'mean_word_length': 4.7346192218157634, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Event-Driven Tactile Learning with Various Location Spiking Neurons

—Tactile sensing is essential for a variety of daily tasks. New advances in event-driven tactile sensors and Spiking Neural Networks (SNNs) spur the research in related ﬁelds. However, SNN-enabled event-driven tactile learning is still in its infancy due to the limited representation abilities of existing spiking neurons and high spatio-temporal complexity in the data. In this paper, to improve the representation capability of existing spiking neurons, we propose a novel neuron model called “location spiking neuron”, which enables us to extract features of event-based data in a novel way. Speciﬁcally, based on the classical Time Spike Response Model (TSRM), we develop the Location Spike Response Model (LSRM). In addition, based on the most commonly-used Time Leaky Integrate-and-Fire (TLIF) model, we develop the Location Leaky Integrate-and-Fire (LLIF) model 1 . By exploiting the novel location spiking neurons, we propose several models to capture the complex spatio-temporal dependencies in the event-driven tactile data. Extensive experiments demonstrate the signiﬁcant improvements of our models over other works on event-driven tactile learning and show the superior energy efﬁciency of our models and location spiking neurons, which may unlock their potential on neuromorphic hardware.


I. INTRODUCTION
With the prevalence of artificial intelligence, computers today have demonstrated extraordinary abilities in the visual and auditory perceptions. Although these perceptions are essential sensory modalities, they may fail to complete tasks in certain situations where tactile perception can help. For example, the visual sensory modality can fail to distinguish objects with similar visual features in less-favorable environments, such as dim-lit or in the presence of occlusions. In such cases, tactile sensing can provide meaningful information like texture, pressure, roughness, or friction and maintain performance. Overall, tactile perception is a vital sensing modality that enables humans to gain perceptual judgment on the surrounding environment and conduct stable movements [1].
With the recent advances in material science and Artificial Neural Networks (ANNs), research on tactile perception has begun to soar, including tactile object recognition [2]- [4], slip detection [5], and texture recognition [6], [7]. Unfortunately, although ANNs demonstrate promising performance on the tactile learning tasks, they are usually power-hungry compared to human brains that require far less energy to perform the tactile perception robustly [8], [9].
Inspired by biological systems, research on event-driven perception has started to gain momentum, and several asynchronous event-based sensors have been proposed, including event cameras [10] and event-based tactile sensors [11]. In contrast to standard synchronous sensors, such event-based sensors can achieve higher energy efficiency, better scalability, and lower latency. However, due to the high sparsity and complexity of event-driven data, learning with these sensors is still in its infancy [12]. Recently, several works [1], [11], [13] utilized Spiking Neural Networks (SNNs) [12], [14], [15] to tackle event-driven tactile learning. Unlike ANNs, which require expensive transformations from asynchronous discrete events to synchronous real-valued frames, SNNs can process event-based sensor data directly. Moreover, unlike ANNs that employ artificial neurons [16]- [18] and conduct real-valued computations, SNNs adopt spiking neurons [19]- [21] and utilize binary 0-1 spikes to process information. This difference reduces the mathematical dot-product operations in ANNs to less computationally expensive summation operations in SNNs. Due to the advantages of SNNs, these works are always energy-efficient and suitable for power-constrained devices. However, due to the limited representative abilities of current spiking neuron models and high spatio-temporal complexity in the event-based tactile data, these works still cannot sufficiently capture spatio-temporal dependencies and thus hinder the performance of event-driven tactile learning.
In this paper, to address the problems mentioned above, we make several contributions that advance event-driven tactile learning.
First, to enable richer representative abilities of existing spiking neurons, we propose a novel neuron model called "location spiking neuron". Unlike existing spiking neuron models that update their membrane potentials based on time steps [22], location spiking neurons update their membrane potentials based on locations. Specifically, based on the Time Spike Response Model (TSRM) [19], we develop the "Location Spike Response Model", henceforth referred to as "LSRM". Moreover, based on the most commonly-used Time Leaky Integrate-and-Fire (TLIF) model [20], we develop the "Location Leaky Integrate-and-Fire" model, henceforth arXiv:2210.04277v2 [cs.NE] 11 Oct 2022 referred to as "LLIF". TSRM and TLIF are the classical SRM and LIF in the literature. We add the character "T (Time)" to highlight their differences from LSRM and LLIF. These location spiking neurons enable the extraction of feature representations of event-based data in a novel way. Previously, SNNs adopted temporal recurrent neuronal dynamics to extract features from the event-based data. With location spiking neurons, we can build SNNs that employ spatial recurrent neuronal dynamics to extract features from the event-based data. We believe location spiking neuron models can have a broad impact on the SNN community and spur research on learning from event sensors like NeuTouch [11], Dynamic Audio Sensors [23], or Dynamic Vision Sensors [10].
Next, we investigate the effectiveness of location spiking neurons and propose two models to capture the complex spatio-temporal dependencies in the event-driven tactile data. Specifically, we build the Hybrid SRM FC that combines a fully-connected SNN with TSRM neurons and a fullconnected SNN with LSRM neurons. Moreover, to capture more spatio-temporal topology knowledge in the data, we propose the Hybrid LIF GNN that fuses spatial and temporal spiking graph neural networks for event-driven tactile learning. To be more specific, the Hybrid LIF GNN first constructs tactile spatial graphs and tactile temporal graphs based on taxel locations and event time sequences, respectively. Then, it utilizes the spatial spiking graph neural network with TLIF neurons and temporal spiking graph neural network with LLIF neurons to extract features of these graphs. Finally, it fuses the spiking tactile features from two networks and provides the final tactile learning prediction. To enable the dynamics of location spiking neurons and optimize the proposed models, we propose three bio-inspired location orders and several loss functions involved with locations. We also propose the timestep-wise inference algorithms for the two models to show their applicability to the temporal data.
Lastly, we apply our proposed models to event-driven tactile learning, including event-driven tactile object recognition and event-driven slip detection, and test them on three challenging tasks. Specifically, the first task requires models to determine the type of objects being handled. The second task requires models to determine the type of containers being handled and the amount of liquid held within, which is more challenging than the first task. And the thrid task asks models to accurately detect the rotational slip ("stable" or "rotate") within 0.15s. Extensive experimental results demonstrate the significant improvements of our models over other state-of-the-art methods on event-driven tactile learning. Moreover, the experiments show that conventional spiking neurons are better at capturing spatial dependencies, while location spiking neurons are better at modeling mid-and-long temporal dependencies. Furthermore, our models outperform ANNs by around 10× to 100× on energy consumption, which shows the superior energy efficiency of our models and may bring new opportunities and unlock their potential on neuromorphic hardware.
Portions of this work "Event-Driven Tactile Learning with Location Spiking Neurons [24]" were accepted by IJCNN 2022 and an oral presentation was given at the IEEE WCCI 2022. In this paper, we expand the idea of location spiking neurons to TLIF neurons and propose LLIF neurons. Moreover, we build the Hybrid LIF GNN for event-driven tactile learning, which adopts spatio-temporal topologies and employs a more complicated network structure compared to the Hybrid SRM FC. We further include more data, experiments, and interpretation to demonstrate the effectiveness and energy efficiency of the proposed neurons and models. Last but not least, we provide preliminary results of the location spiking neurons on event-driven audio learning to show the broad applicability and potential impact of this work.
The rest of the paper is organized as follows. In Section II, we provide an overview of related work on SNNs and eventdriven tactile sensing and learning. In Section III, we start by introducing notations for existing spiking neurons and extend them to the specific location spiking neurons. We then propose various models with location spiking neurons for event-driven tactile learning. Last, we provide implementation details and algorithms related to the proposed models. In Section IV, we demonstrate the effectiveness and energy efficiency of our proposed models on benchmark datasets. Finally, we discuss the broad impact of this work and conclude in Section V.

II. RELATED WORK
In the following, we provide a brief overview of related work on SNNs and event-driven tactile sensing and learning.

A. Spiking Neural Networks (SNNs)
With the prevalence of Artificial Neural Networks (ANNs), computers today have demonstrated extraordinary abilities in many cognition tasks. However, ANNs only imitate brain structures in several ways, including vast connectivity and structural and functional organizational hierarchy [22]. The brain has more information processing mechanisms like the neuronal and synaptic functionality [25], [26]. Moreover, ANNs are much more energy-consuming than human brains. To integrate more brain-like characteristics and make artificial intelligence models more energy-efficient, researchers propose Spiking Neural Networks (SNNs), which can be executed on power-efficient neuromorphic processors like TrueNorth [27] and Loihi [28]. Similar to ANNs, SNNs can adopt general network topologies like convolutional layers and fully-connected layers, but use different neuron models [21]. Commonlyused neuron models for SNNs are the Time Leaky Integrateand-Fire (TLIF) model [20] and the Time Spike Response Model (TSRM) [19]. Due to the non-differentiability of these spiking neuron models, it still remains challenging to train SNNs. Nevertheless, several solutions have been proposed, such as converting trained ANNs to SNNs [29], [30] and approximating the derivative of the spike function [31], [32]. In this work, we propose location spiking neurons to enhance the representative abilities of existing spiking neurons. These location spiking neurons maintain the spiking characteristic but employ the spatial recurrent neuronal dynamics, which enable us to build energy-efficient SNNs and extract features of event-based data in a novel way. Moreover, based on the optimization methods for SNNs with existing spiking neurons, A transmission delay may be included in the definition of ij (·). This figure is adopted from [24].
we can derive the approximate backpropagation methods for SNNs with location spiking neurons.

B. Event-Driven Tactile Sensing and Learning
With the prevalence of material science and robotics, several tactile sensors have been developed, including non-eventbased tactile sensors like the iCub RoboSkin [33] and the SynTouch BioTac [34] and event-driven tactile sensors like the NeuTouch [11] and the NUSkin [35]. In this paper, we focus on event-driven tactile learning with SNNs. Since the development of event-driven tactile sensors is still in its infancy [13], little prior work exists on learning event-based tactile data with SNNs. The work [1] employed a neural coding scheme to convert raw tactile data from non-event-based tactile sensors into event-based spike trains. It then utilized an SNN to process the spike trains and classify textures. A recent work [11] released the first publicly-available event-driven visual-tactile dataset collected by NeuTouch and proposed an SNN based on SLAYER [14] to solve the event-driven tactile learning. Moreover, to naturally capture the spatial topological relations and structural knowledge in the event-based tactile data, a very recent work [13] adopted the spiking graph neural network [15] to process the event-based tactile data and conduct the tactile object recognition. In this paper, different from previous works building SNNs with spiking neurons that employ the temporal recurrent neuronal dynamics, we construct SNNs with location spiking neurons to capture the complex spatio-temporal dependencies in the event-based tactile data and improve the event-driven tactile learning.

III. METHODS
In this section, we first demonstrate the spatial recurrent neuronal dynamics of location spiking neurons by introducing notations for the existing spiking neuron models and extending them to the location spiking neuron models. We then introduce various models with location spiking neurons for event-driven tactile learning. Last, we provide implementation details and algorithms related to the proposed models.
A location delay may be included in the definition of ij (·). "<" and "≥" indicate the location order. This figure is adopted from [24].

A. TSRM vs. LSRM
Spiking neuron models are mathematical descriptions of specific cells in the nervous system. They are the basic building blocks of SNNs. Two commonly-used spiking neuron models are the TLIF and the TSRM [36]. In this section, we introduce the TSRM and transform it to a location spiking neuron model -the LSRM.
In the TSRM, the temporal recurrent neuronal dynamics of neuron i are described by its membrane potential u i (t). When u i (t) exceeds a predefined threshold ϑ at the firing time t (f ) i , the neuron i will generate a spike. The set of all firing times of neuron i is denoted by where t (n) i is the most recent spike time t (f ) i < t. The value of u i (t) is governed by two different spike response processes: is the refractory kernel, which describes the response of neuron i to its own spikes at time t. ij (·) is the incoming spike response kernel, which models the neuron i's response to the presynaptic spikes from neuron j at time t. w ij accounts for the connection strength between neuron i and neuron j and scale the incoming spike response. Figure 1(a) visualizes the refractory dynamics and Figure 1(b) visualizes the incoming spike dynamics. Without loss of generality, such temporal recurrent neuronal dynamics also apply to other spiking neuron models, such as the TLIF. From the above descriptions, we find that existing spiking neuron models have explicit temporal recurrence but do not possess explicit spatial recurrence, which, to some extent, limits their representative abilities.
To enrich the representative abilities of existing spiking neuron models, we propose location spiking neurons, which adopt the spatial recurrent neuronal dynamics and update their membrane potentials based on locations 2 . These neurons exploit explicit spatial recurrence and enable us to extract features of event-based data in a novel way. Specifically, we adopt the TSRM and transform it to the LSRM. In the LSRM, the spatial recurrent neuronal dynamics of neuron i are described by its location membrane potential u i (l). When u i (l) exceeds a predefined threshold ϑ at the firing location l (f ) i , the neuron i will generate a spike. The set of all firing locations of neuron i is denoted by where l (n) i is the nearest firing location l (f ) i < l. "<" indicates the location order, which is manually set and will be discussed in Section III-E. The value of u i (l) is governed by two different spike response processes: is the refractory kernel, which describes the response of neuron i to its own spikes at location l. ij (·) is the incoming spike response kernel, which models the neuron i's response to the presynaptic spikes from neuron j at location l. w ij accounts for the connection strength between neuron i and neuron j and scale the incoming spike response. Figure 2(a) visualizes the refractory dynamics of LSRM neurons and Figure 2(b) visualizes the incoming spike dynamics of LSRM neurons. The threshold ϑ of LSRM neurons can be different from that of TSRM neurons, while we set the same for simplicity.

B. TLIF vs. LLIF
The TLIF model is the most commonly-used spiking neuron model since it is computationally tractable and maintains biological fidelity to a certain degree [31]. A similar transformation process can be applied to the TLIF to derive its corresponding location spiking neuron model -the LLIF.
The dynamics of the TLIF neuron i are governed by where u i (t) represents the internal membrane potential of the neuron i at time t, τ is a time constant, and I(t) signifies the pre-synaptic input obtained by the combined action of synaptic weights and pre-neuronal activities. To better understand the membrane potential update of TLIF neurons, the Euler method is used to transform the first-order differential equation of Eq. (5) into a recursive expression: (b) the spatial recurrent neuronal dynamics of a LLIF neuron i. Both of them take as input binary spikes and output binary spikes. x j and x j represents the input signal to the neuron i from neuron j, u i is the neuron's membrane potential, and o i is the neuron's output. An output spike will be emitted from the neuron when its membrane potential surpasses the firing threshold u th , after which the membrane potential will be reset to ureset.
where j w j x j (t) is the weighted summation of the inputs from pre-neurons at the current time step. Equation (6) can be further simplified as: where α = 1 − dt τ can be considered a decay factor, and w j is the weight incorporating the scaling effect of dt τ . When u i (t) exceeds a certain threshold u th , the neuron emits a spike, resets its membrane potential to u reset , and then accumulates u i (t) again in subsequent time steps. Figure 3(a) visualizes the dynamics of a TLIF neuron. From the above descriptions, like the TSRM, we find that the TLIF also has explicit temporal recurrence but does not possess explicit spatial recurrence, which, to some extent, limits its representation abilities.
To enrich the representation abilities of the TLIF, we propose the LLIF. Different from the temporal dynamics shown in Eq. (5), the LLIF neuron i employs the spatial dynamics: where u i (l) represents the internal membrane potential of a LLIF neuron i at location l, τ is a location constant, and I(l) represents the pre-synaptic input. To better understand the membrane potential update of LLIF neurons, we use the Euler method to transform the first-order differential equation of Eq. (8) into a recursive expression: where j w j x j (l) is the weighted summation of the inputs from pre-neurons at the current location. Equation (9) can be further simplified as: where β = 1 − dl τ can be considered a location decay factor, and w j is the weight incorporating the scaling effect of dl τ . When u i (l) exceeds a certain threshold u th , the neuron emits a spike, resets its membrane potential to u reset , and processes the transposed input spikes X in and employs the spatial recurrent neuronal dynamics (shown with purple dashed arrows) of LSRM neurons to extract features from the data. Finally, the spiking representations from two networks are concatenated to yield the final predicted label. (32) and (20) represent the sizes of fully-connected layers, where we assume the number of classes (K) to be equal to 20. This figure is adapted from [24]. then accumulates u i (l) again at subsequent locations. u th and u reset of LLIF neurons can be different from those of TLIF neurons, while we set the same for simplicity. The location order is manually set and will be discussed in Section III-E. Figure 3(b) visualizes the spatial recurrent neuronal dynamics of a LLIF neuron.

C. Event-Driven Tactile Learning with the LSRM Neurons
To boost the event-based tactile learning performance, we propose models with location spiking neurons, which capture complex spatio-temporal dependencies in the event-based tactile data. In this paper, we focus on processing the data collected by NeuTouch [11], a biologically-inspired eventdriven fingertip tactile sensor with 39 taxels arranged spatially in a radial fashion (see Fig. 4). In this section, we introduce the event-driven tactile learning with the LSRM neurons.
1) Hybrid SRM FC: Figure 4 presents the network structure of the Hybrid SRM FC. From the figure, we can see that the model has two components, including the SNN with TSRM neurons (SNN TSRM) and the SNN with LSRM neurons (SNN LSRM). Specifically, SNN TSRM employs the temporal recurrent neuronal dynamics to extract spiking feature representations from the event-based tactile data X in ∈ R N×T , where N is the total number of taxels and T is the total time length of event sequences. SNN LSRM utilizes the spatial recurrent neuronal dynamics to extract spiking feature representations from the event-based tactile data X in ∈ R T×N , where X in is transposed from X in . The spiking representations from two networks are then concatenated to yield the final task-specific output.
2) SNN TSRM vs. SNN LSRM: The network structure of SNN TSRM is shown in the top part of Fig. 4. It employs two spiking fully-connected layers with TSRM neurons (SFc) to process X in and generate the spiking representations O 1 ∈ R K×T , where K is the output dimension determined by the task. The membrane potential u i (t), the output spiking state o i (t), and the set of all firing times F i of TSRM neuron i in SFc are decided by: where w ij are the trainable parameters, η(·) and (·) are predefined by hyperparameters, Γ i is the set of presynaptic neurons spanning over the spatial domain, which is utilized to capture the spatial dependencies in the event-based data. The network structure of SNN LSRM is shown in the bottom part of Fig. 4. It employs two spiking fully-connected layers with LSRM neurons (SFc-location) to process X in and generate the spiking representations O 2 ∈ R K×N , where K is the output dimension decided by the task. The membrane potential u i (l), the output spiking state o i (l), and the set of all firing locations G i of LSRM neuron i in SFc-location are:  [13]. Each circle represents a taxel of NeuTouch. (b) two different tactile temporal graphs Gt for a specific taxel n = 1: the above one is the sparse tactile temporal graph, while the below one is the dense tactile temporal graph.
where w ij are the trainable parameters, η (·) and (·) are predefined by hyperparameters, Γ i is the set of presynaptic neurons spanning over the temporal domain, which is utilized to model the temporal dependencies in the event-based data. Such location spiking neurons tap the representative potential and enable us to capture features in this way.

3) Concatenate:
We concatenate the spiking representations of O 1 and O 2 along the last dimension and obtain the final output spike train O ∈ R K×(T +N ) . The predicted label is associated with the neuron k ∈ K with the largest number of spikes in the domain of T + N .

D. Event-Driven Tactile Learning with the LLIF Neurons
In this section, we utilize the LLIF neurons to propose the Hybrid LIF GNN, which fuses spatial and temporal spiking graph neural networks and captures complex spatio-temporal dependencies in the event-based tactile data.
1) Tactile Graph Construction: Given event-based tactile inputs X in ∈ R N ×T , we construct tactile spatial graphs and tactile temporal graphs as illustrated in Fig. 5.
The tactile spatial graph G s (t) = (V t , E t ) at time step t explicitly captures the spatial structual information in the data, while the tactile temporal graph G t (n) = (V n , E n ) for a specific taxel n explicitly models the temporal dependency in the data. V t = {v t n |n = 1, ..., N } and V n = {v t n |t = 1, ..., T } represent nodes of G s (t) and G t (n), respectively, and the attribute of v t n is the event feature of the n-th taxel at time step t. E t = {e t i,j |i, j = 1, ..., N } represents the edges of G s (t), where e t i,j ∈ {0, 1} indicates whether the nodes v t i , v t j are connected (denoted as 1) or disconnected (denoted as 0). E t is formed by the Minimum Spanning Tree (MST) algorithm, where the Euclidean distance between taxels is used to determine whether the edges are in the MST. Since the 2D coordinates (x, y) of taxels do not change with time, E t remains the same throughout time. Moreover, the adjacency matrix of E t is symmetric (i.e., the edges are indirect) as we assume the mutual spatial dependency in the data. E n = {e p,q n |p, q = 1, ..., T } represents the edges of G t (n), where e p,q n ∈ {0, 1} and each edge is direct. Based on different temporal dependency assumptions, we propose two kinds of tactile temporal graphs shown in Fig. 5(b). One is sparse since we assume the current state only directly impacts the nearest future state. While the other is dense since we assume the current state has a broad impact on the future states. E n remains the same for all N taxels.
2) Hybrid LIF GNN: To process the data from our tactile graphs and capture the complex spatio-temporal dependencies in the event-based tactile data, we propose the Hybrid LIF GNN (Fig. 6), which fuses spatial and temporal spiking graph neural networks. Specifically, we adopt the spatial spiking graph neural network (SSGNN) with TLIF neurons [13] to process the tactile spatial graphs. And we use temporal recurrent neuronal dynamics to extract features from the tactile spatial graphs. Moreover, we develop the temporal spiking graph neural network (TSGNN) with LLIF neurons to process the tactile temporal graphs. And we use spatial recurrent neuronal dynamics to extract features from the tactile temporal graphs. Finally, we fuse the spiking features from two networks and obtain the final prediction.
a) SSGNN vs. TSGNN: SSGNN takes as input tactile spatial graphs, while TSGNN takes as input tactile temporal graphs. SSGNN has one spatial spiking graph (SSG) layer and three spatial spiking fully-connected (SSFc) layers, where TLIF neurons that employ the temporal recurrent neuronal dynamics are the basic building blocks. On the other hand, TSGNN has one temporal spiking graph (TSG) layer and three temporal spiking fully-connected (TSFc) layers, where LLIF neurons that possess the spatial recurrent neuronal dynamics are the basic building blocks.
b) SSG vs. TSG: Based on Eq. (7), the membrane potential u i (t) and output spiking state o i (t) of TLIF neuron i in SSG are decided by: where I(t) = GNN(G s (t)) is to capture the spatial structual information. Based on Eq. (10), the membrane potential u i (l) and output spiking state o i (l) of LLIF neuron i in TSG are decided by: where I(l) = GNN(G t (l)) is to model the temporal dependencies in the data. l is represented by the taxel n ∈ N in this paper. To fairly compare with other baselines, we use TAGConv [37] as GNN in this paper. c) SSFc vs. TSFc: The membrane potential u i (t) and output spiking state o i (t) of TLIF neuron i in SSFc are decided by Eq. (13), where I(t) = Fc(P re(t)) and P re(t) is the previous layer's output at time step t. While the membrane potential u i (l) and output spiking state o i (l) of LLIF neuron i in TSFc are decided by Eq. (14), where I(l) = Fc(P re(l)) and P re(l) is the previous layer's output at location l (taxel n).
where O 1 (t) ∈ R K . TSGNN extracts spiking features O 2 ∈ R K×N . It predicts its label vector O 2 ∈ R K by averaging O 2 over the spatial window N , where O 2 (l) ∈ R K . Then, to fuse predictions from these two networks, we take the mean or element-wise max of these two label vectors O 1 and O 2 and obtain the final predicted label vector O ∈ R K . The predicted label is associated with the neuron with the largest value.
E. Implementations and Algorithms 1) Location Orders: To enable location spiking neurons' spatial recurrent neuronal dynamics, we propose three location orders for event-based tactile learning (Fig. 7) based on three major fingerprint patterns of humans -arch, whorl, and loop. Three examples are shown here. Each number in the brackets represents the taxel index, see Fig. 5(a).
• An example for the arch-like location order: [11, 25, [11], [14], we propose a location spike-count loss to optimize the SNN with LSRM neurons: which captures the difference between the observed output spike count N l=0 o k (l) and the desired spike count N l=0ô k (l) across the K neurons. Moreover, to optimize the Hybrid SRM FC, we develop a weighted spike-count loss: which first balances the contributions from two SNNs and then captures the difference between the observed balanced output spike count c=0ô k (c) across the K output neurons. For both L LSRM and L 1 , the desired spike counts have to be specified for the correct and incorrect classes and are task-dependent hyperparameters. We set these hyperparameters as in [11]. To overcome the non-differentiability of spikes and apply the backpropagation algorithm, we use the approximate gradient proposed in SLAYER [14]. The timestep-wise inference algorithm of the Hybrid SRM FC is shown in Algorithm 1. And the corresponding timestep-wise training algorithm can be derived by incorporating the weighted spike-count loss.
Algorithm 1 Timestep-wise inference algorithm of the Hybrid SRM FC, adopted from [24] Input: event-based tactile inputs X in ∈ R N ×T , N taxels, and the total time length T .
To overcome the non-differentiability of spikes and apply the backpropagation algorithm, we use the rectangular function [31] to approximate the derivative of the spike function (Heaviside function) in Eqs. (13) and (14). The timestep-wise inference algorithm of the model is shown in Algorithm 2.
Algorithm 2 Timestep-wise inference algorithm of the Hybrid LIF GNN Input: event-based tactile inputs X in ∈ R N ×T , N taxels, and the total time length T Output: timestep-wise label vectors of O 1 , O 2 , and O 1: for t ← 1 to T do 2: form t tactile spatial graphs G s with X ∈ R N ×t 3: form N tactile temporal graphs G t withX 5: for i ← 1 to t do  for j ← 1 to N do max can be used 15: end for

IV. EXPERIMENTS
In this section, we extensively evaluate our proposed models. Specifically, we first conduct experiments on the Hybrid SRM FC built with TSRM and LSRM neurons. Then, we demonstrate the superior performance of the Hybrid LIF GNN built with TLIF and LLIF neurons. Finally, we compare the two models on the benchmark datasets. The source code and experimental configuration details are available at https://github.com/pkang2017/TactileLSN.

A. Hybrid SRM FC
In this section, we first introduce the datasets and models for the experiments. Next, to show the effectiveness of the Hybrid SRM FC, we extensively evaluate its performance on the benchmark datasets and compare it with state-of-the-art models. Finally, we demonstrate the superior energy efficiency of Hybrid SRM FC over ANNs and show the high-efficiency benefit of LSRM neurons. We utilize the slayerPytorch framework 3 to implement the proposed model and employ RMSProp with the l 2 regularization to optimize them. 1) Datasets: We use the datasets collected by NeuTouch [11], including "Objects-v1" and "Containers-v1" for event-driven tactile object recognition and "Slip Detection" for event-driven slip detection. Unlike "Objects-v1" only requiring models to determine the type of objects being handled, "Containers-v1" asks models about the type of containers being handled and the amount of liquid (0%, 25%, 50%, 75%, 100%) held within. Thus, "Containers-v1" is more challenging for event-driven tactile object recognition. Moreover, the task of event-driven slip detection is also challenging since it requires models to detect the rotational slip within a short time, like 0.15s for "Slip Detection". We summarize the dataset statistics in Table I, where TD is for time durations, SSR is for spike sampling rates, T =TD / SSR is the total time length, N is the total number of taxels (each tactile sensor has 39 taxels and two tactile sensors are used), and K is the number of classes. We split the data into a training set (80%) and a test set (20%) with an equal class distribution in the experiments. We repeat each experiment for five rounds and report the average accuracy.
2) Comparing Models: We compare our model with the state-of-the-art SNN methods for event-driven tactile learning, including Tactile-SNN [11] and TactileSGNet [13]. Tactile-SNN employs TSRM neurons as the building blocks, and the network structure of Tactile-SNN is Input-SFc0-SFc1. TactileSGNet utilizes TLIF neurons as the building blocks and proposes the spiking graph neural network (SGNet). *These values come from [11]. The best performance is in bold.
The network structure of TactileSGNet is Input-SGNet-SFc1-SFc2-SFc3. As in [11], we also compare our model against conventional deep learning, specifically Gated Recurrent Units (GRUs) [38] with Multi-layer Perceptrons (MLPs) and 3D convolutional neural networks [39]. The network structure of GRU-MLP is Input-GRU-MLP, where MLP is only utilized at the final time step. And the network structure of CNN-3D is Input-3D CNN1-3D CNN2-Fc1.
3) Basic Performance: Table II presents the test accuracies on the three datasets. We observe that the Hybrid SRM FC significantly outperforms the state-of-the-art SNNs. The reason why our model is superior to other SNNs could be twofold: (1) different from state-of-the-art SNNs that only extract features with existing spiking neurons, our model employs an SNN with location spiking neurons to extract features in a novel way; (2) our model fuses SNN TSRM and SNN LSRM to better capture complex spatio-temporal dependencies in the data. We also compare our model with ANNs, which provide fair comparison baselines for fully ANN architectures since they employ similar lightsome network architectures as ours. From Table II, we find out that our model outperforms ANNs on the three tasks, which might be because our model is more compatible with various kinds of event-based data and better maintains the sparsity to prevent overfitting.

4) Ablation Studies:
To examine the effectiveness of each component in the proposed model, we seperately train SNN TSRM (which is exactly Tactile-SNN) and SNN LSRM (which is referred to as Location Tactile-SNN). From Table III, we surprisingly find out that Location Tactile-SNN significantly surpasses Tactile-SNN on the datasets for eventdriven tactile object recognition and provides comparable performance on the event-driven slip detection. The reason for this could be two-fold: (1) the time durations of event-driven tactile object recognition datasets are longer than that of "Slip Detection", and Location Tactile-SNN is good at capturing the mid-and-long term dependencies in these object recognition datasets; (2) like Tactile-SNN, Location Tactile-SNN can still capture the spatial dependencies in the event-driven tactile data ("Slip Detection") due to the spatial recurrent neuronal dynamics of location spiking neurons. Furthermore, we examine the sensitivities of λ in Eq.(18) and location orders. From Table III, we notice the results of related models are close, proving that the λ tuning and location orders do not significantly impact the task performance. 5) Timestep-wise Inference: We evaluate the timestep-wise inference performance of the Hybrid SRM FC and validate the contributions of the two components in it. Moreover, we propose a time-weighted Hybrid SRM FC to better balance the two components' contributions and achieve the better overall performance. Figure 8 , where the hyperparameter ψ controls the balance between SNN TSRM's contribution and SNN LSRM's contribution and T is the total time length. From the figures, we can see that SNN TSRM has good "early" accuracies on the three tasks since it well captures the spatial dependencies with the help of Eq. (11). However, its accuracies do not improve too much at the later stage since it does not sufficiently capture the temporal dependencies. In contrast, SNN LSRM has fair "early" accuracies, while its accuracies jump a lot at the later stage since it models the temporal dependencies in Eq. (12). The Hybrid SRM FC adopts the advantages of these two components and extracts spatio-temporal features from various views, which enables it to have a better overall performance. Furthermore, after employing the time-weighted output and shifting more weights to SNN TSRM at the early stage, the time-weighted Hybrid SRM FC can have a good "early" accuracy as well as an excellent "final" accuracy. 6) Energy Efficiency: Following the estimation method in [15], [40] 4 , we estimate the computational costs of the Hybrid SRM FC and ANNs on the three datasets. As shown in Fig. 8(d), our proposed model has no multiplication operations and achieves far fewer addition operations than ANN models on the three datasets. Moreover, based on Table IV, the compression ratio of total operations (ANNs Opts. / Ours Opts.) is between 9.80× and 121.78×. These results are consistent with the fact that the sparse spike communication and eventdriven computation underlie the efficiency advantage of SNNs Note that we use the same event sequences as [11] and the first spike occurs at around 2.0s for "Objects-v1" and "Containers-v1". (d) Operation cost (×10 6 ) comparison between the Hybrid SRM FC and ANNs on the benchmark datasets. (e) Operation cost (×10 6 ) and percentage comparison between SNN TSRM and SNN LSRM on the benchmark datasets. and demonstrate the potentials of our model on neuromorphic hardware. We further compare the costs of SNN TSRM and SNN LSRM on the benchmark datasets. From Fig. 8(e), we can see that the cost of SNN LSRM is almost equal to that of SNN TSRM on each dataset, which shows that the location spiking neurons (LSRM neurons) have the similar energy efficiency compared to existing spiking neurons (TSRM neurons). Such high-efficiency benefits make location spiking neurons a perfect fit for neuromorphic hardware.

B. Hybrid LIF GNN
In this section, to fairly compare with other published models with TLIF neurons [13], we evaluate the Hybrid LIF GNN on two real-world datasets -"Objects-v0" and "Containers-v0". 5 Then, we conduct several ablation studies to examine the effectiveness of some designs in the Hybrid LIF GNN. Finally, we demonstrate the superior energy efficiency of our model over Graph Neural Networks (GNNs) and show the high-efficiency benefits of location spiking neurons. We utilize the Pytorch framework to implement the models and employ Adam with the decay learning rate to optimize them. 5 In this section, to be consistent with [13], we use accuracies (%).

1) Datasets:
To fairly compare with other published models with TLIF neurons [13], we evaluate the Hybrid LIF GNN built with TLIF and LLIF neurons on "Objects-v0" and "Containers-v0". These two datasets are the initial versions of "Objects-v1" and "Containers-v1". And we summarize the dataset statistics in Table V. We split the data into a training set (80%) and a test set (20%) with an equal class distribution in the experiments. We repeat each experiment for five rounds and report the average accuracy. Interested readers can find more details about the datasets in [13] and the corresponding website 6 . 2) Comparing Models: We compare the Hybrid LIF GNN with the state-of-the-art methods on event-based tactile object recognition, including the TactileSGNet series [13] and Tactile-SNN [11]. All the models in the TactileSGNet series adopt the neural networks with TLIF neurons. More precisely, *These values come from [13]. And the value for Tactile-SNN comes from [11]. ‡ represents Hybrid LIF GNN-sparse-mean-loop, where we use "sparse tactile temporal graph", "mean fusion", and "loop-like location order". The best performance is in bold.
the network structure for TactileSGNet-CNN is Input-Spiking CNN-SSFc1-SSFc2-SSFc3, where the tactile data are organized in a grid structure according to the spatial distribution of taxels, and the CNN with TLIF neurons is utilized to extract features from this grid. The network structure for TactileSGNet-MLP is Input-SSFc0-SSFc1-SSFc2-SSFc3, where SSFc0 (the FC layer with TLIF neurons) is utilized to extract features from input data. The network structure for TactileSGNet-GCN is Input-Spiking GCN-SSFc1-SSFc2-SSFc3, where the tactile spatial graphs are built, and Spiking GCN [41] is utilized to extract features from the graphs. The network structure for TactileSGNet-TAGConv is Input-Spiking TAGConv-SSFc1-SSFc2-SSFc3 (exactly the same as SSGNN of our model), where the tactile spatial graphs are built, and TAGConv [37] with TLIF neurons is utilized to extract features from the graphs. Tactile-SNN employs TSRM neurons as the building blocks, which is the SNN TSRM in Fig. 4. Please note that Tactile-SNN [11] utilized the data of both hands (N = 78) in their paper, while other models, including ours, use the data of one hand (N = 39). We also compare the Hybrid LIF GNN against GNNs. Specifically, the GNNs have the same network structures as the Hybrid LIF GNN, including one recurrent TAGConv-Fc-Fc-Fc for T tactile spatial graphs, one recurrent TAGConv-Fc-Fc-Fc for N tactile temporal graphs, and one fusion module to fuse the predictions from two branches. The major difference between our model and GNNs is that GNNs employ artificial neurons and adopt different activation functions in Eqs. (13) and (14).
Our model utilizes Heaviside function as its activation function while GNNs utilize ANNs' activation functions.

3) Basic Performance:
We report the test accuracies on the two event-driven tactile object recognition datasets in Table VI. From this table, we can see that the Hybrid LIF GNN significantly outperforms state-of-the-art methods including TactileSGNet series [13] and Tactile-SNN [11]. The reason why our model can achieve the better performance could be two-fold: (1) different from state-of-the-art methods that only utilize TLIF neurons to extract features from the tactile spatial graphs, our model also employs TSGNN with LLIF neurons to extract features from the tactile temporal graphs; (2) our model fuses SSGNN and TSGNN to capture complex spatio-temporal dependencies in the data. We also compare our model with GNNs by replacing the spike functions in Eqs. (13) and (14) with activation functions, such as linear, elu, or LeakyRelu. These models provide fair comparison baselines for fully GNN architectures since they employ the same network architecture as ours. From Table VI, we observe that the Hybrid LIF GNN outperforms GNNs on the two datasets, which might be because our model is more compatible with event-based data and better maintains the sparsity to prevent overfitting.

4) Ablation Studies:
We further provide ablation studies for exploring the optimal design choices. From Table VII, we find out that the combination of "sparse tactile temporal graph" and "mean fusion" performs better than other combinations. The reason for this could be two-fold: (1) the dense tactile temporal graph involves too many insignificant temporal dependencies and does not differentiate the importance of each dependency; (2) the max fusion results in information loss. From Table VII, we also notice that the results of three bio-inspired location orders are close, proving that the location order does not significantly impact the task performance. 5) Timestep-wise Inference: Figure 9 shows the timestepwise inference accuracies (%) for SSGNN, TSGNN, the Hybrid LIF GNN, and the time-weighted Hybrid LIF GNN on the two datasets. Specifically, the output of time-weighted Hybrid LIF GNN at time t is where ζ controls the balance between SSGNN's contribution and TSGNN's contribution and T is the total time length. From the figure, we can see that SSGNN has a good "early" accuracy with the help of tactile spatial graphs, while its accuracy does not improve too much at the later stage since it cannot well capture the temporal dependencies. In contrast, TSGNN has a fair "early" accuracy, while its accuracy jumps a lot at the later stage since it models the temporal dependencies explicitly. The Hybrid LIF GNN adopts the advantages of these two models and extracts spatio-temporal features from multiple views, which enables it to have a better overall performance. Furthermore, after employing the time-weighted output and set ζ = 2 to shift more weights to SSGNN at the early stage, the time-weighted model can have a good "early" accuracy as well as an excellent "final" accuracy, see red lines in Fig. 9. 6) Energy Efficiency: Following the estimation method in [15], we estimate the computational costs of Hybrid LIF GNN-sparse-mean-loop and GNNs from Table VI on the two datasets. As shown in Fig. 10(a), our model has  We further compare the costs of SSGNN and TSGNN. From Fig. 10(b), we can see that the cost of TSGNN only accounts for 21% ∼ 23% of total costs. The reason why TSGNN can achieve higher energy efficiency could be two-fold: (1) the sparse tactile temporal graphs require fewer operations in the TSG layer; (2) the number of taxels N (39 locations) is much less than the number of time steps T (250 for "Objects-v0" and 325 for "Containers-v0"), which leads to less recurrent neuronal dynamics in TSGNN. Such high-efficiency benefits make TSGNN and location spiking neurons (LLIF neurons) a perfect fit for neuromorphic hardware.

C. Comparison
We further compare the Hybrid SRM FC (Fig.4) and the Hybrid LIF GNN (Fig.6) on "Objects-v1", "Containers-v1", and "Slip Detection". From Table VIII, we can see that the Hybrid LIF GNN outperforms the Hybrid SRM FC on "Objects-v1" and "Containers-v1" and they both achieve the perfect slip detection. The reason for this is that the Hybrid LIF GNN adopts graph topologies and has a more complicated structure than the Hybrid SRM FC.

V. DISCUSSION AND CONCLUSION
This paper proposes a novel neuron model -"location spiking neuron". Based on the neuronal dynamic equations of conventional spiking neurons and location spiking neurons, we can see that both of them can extract spatio-temporal dependencies from the data. Specifically, the conventional spiking neurons employ the temporal recurrent neural dynamics to update their membrane potentials and capture spatial dependencies by aggregating the information from presynaptic neurons, see Eqs. (2), (7), (11) and (13). While location spiking neurons use spatial recurrent neural dynamics to update their potentials and model temporal dependencies by aggregating the information from presynaptic neurons, see Eqs. (4), (10), (12) and (14). Moreover, based on experimental results, we can see that conventional spiking neurons (in SNN TSRM and SSGNN) are better at capturing spatial dependencies which benefit the "early" accuracy, while location spiking neurons (in SNN LSRM and TSGNN) are better at modeling mid-andlong temporal dependencies which benefit the "late" accuracy. Networks built only with conventional spiking neurons or networks built only with location spiking neurons cannot sufficiently capture spatio-temporal dependencies in the eventbased tactile data. Thus, we always concatenate or fuse the networks to sufficiently capture spatio-temporal dependencies in the data and boost the task performance.
By introducing LSRM neurons and LLIF neurons, we verify that the idea of location spiking neurons can be applied to various existing spiking neuron models like TSRM neurons and TLIF neurons and strengthen their feature representation abilities. Moreover, we extensively evaluate the models built with these novel neurons and demonstrate their superior performance and energy efficiency. Furthermore, by comparing the Hybrid LIF GNN with the Hybrid SRM FC, we show that the location spiking neurons can be utilized to build more complicated models to further improve the task performance.
Besides event-driven tactile learning, we can apply the models with location spiking neurons to other event-driven learning fields. To show the broad imapct of location spiking neurons, we apply the Hybrid SRM FC (Fig. 4) to event-driven audio learning and provide preliminary results 7 . Specifically, we use the N-TIDIGITS18 dataset [23], which is collected by playing the audio files from the TIDIGITS dataset [42] to the dynamic audio sensor -the CochleaAMS1b sensor [43]. The dataset includes both single digits and connected digit sequences. We use the single digit part of the dataset, which consists of 11 categories, including 'oh', 'zero', and digits '1' to '9'. The single digit dataset has 4,950 samples, of which 2,486   samples are utilized for testing. A spike audio sample of digit '2' is shown in Fig. 11, where x-axis indicates the event time, and y-axis indicates the 64 frequency channels of the CochleaAMS1b sensor. Each blue dot in the sample represents an event that occurs at time t e and frequency f e . In this application, we regard "frequency channels" as "locations" and apply the Hybrid SRM FC to process the spike audio inputs, see Fig. 11. Table IX presents the test accuracies on the single digit dataset. We observe that our model significantly outperforms SNN TSRM 8 . Moreover, figure 12 shows the training and testing profiles for SNN TSRM and our model on the dataset. From this figure, we can see that our model converges faster and attains a lower loss and a higher accuracy compared to SNN TSRM.
These preliminary results show that location spiking neurons 8 Please note that there are 2,486 testing samples. This means 1% improvement leads to 24.86 more samples are correctly classified.
can be applied to other event-driven learning fields. Moreover, location spiking neurons can extract useful features from the data and benefit the task.
In this work, we propose a novel neuron model -"location spiking neuron". Specifically, we introduce two concrete location spiking neurons -the LSRM neurons and LLIF neurons. We demonstrate the spatial recurrent neuronal dynamics of these neurons and compare them with the conventional spiking neurons -the TSRM neurons and TLIF neurons. By exploiting these location spiking neurons, we develop several models for event-driven tactile learning to sufficiently capture the complex spatio-temporal dependencies in the data. The extensive experimental results on the event-driven tactile datasets demonstrate the extraordinary performance and high energy efficiency of our models and location spiking neurons. This further unlocks their potential on neuromorphic hardware. Overall, this work sheds new light on SNN representation learning and eventdriven learning, which may facilitate the understanding of advanced cognitive intelligence.

ACKNOWLEDGMENTS
Portions of this work "Event-Driven Tactile Learning with Location Spiking Neurons [24]" were accepted by IJCNN 2022 and orally presented at the IEEE WCCI in 2022.