{'fraction_non_alphanumeric': 0.03648543573180214, 'fraction_numerical': 0.013637179257132604, 'mean_word_length': 4.538092083471348, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Spiking Neural Networks with Single-Spike Temporal-Coded Neurons for Network Intrusion Detection

Spiking neural network (SNN) is interesting due to its strong bio-plausibility and high energy efficiency. However, its performance is falling far behind conventional deep neural networks (DNNs). In this paper, considering a general class of single-spike temporal-coded integrate-and-fire neurons, we analyze the input-output expressions of both leaky and nonleaky neurons. We show that SNNs built with leaky neurons suffer from the overly-nonlinear and overly-complex input-output response, which is the major reason for their difficult training and low performance. This reason is more fundamental than the commonly believed problem of nondifferentiable spikes. To support this claim, we show that SNNs built with nonleaky neurons can have a less-complex and less-nonlinear input-output response. They can be easily trained and can have superior performance, which is demonstrated by experimenting with the SNNs over two popular network intrusion detection datasets, i.e., the NSL-KDD and the AWID datasets. Our experiment results show that the proposed SNNs outperform a comprehensive list of DNN models and classic machine learning models. This paper demonstrates that SNNs can be promising and competitive in contrast to common beliefs.


I. INTRODUCTION
Spiking neural networks (SNNs) have great theoretical significance because of their bio-inspiration nature. They work more like biological neurons than conventional deep neural networks (DNNs). Neurons communicate via spike waveform just as biological neurons and work asynchronously, i.e., generate output spikes without waiting for all input neurons to spike. SNNs are promising practically mainly because of their strong potential of achieving high energy efficiency, which is important for mobile and IoT applications involving large and deep neural networks. The asynchronous working principle potentially leads to low latency that is attractive for high-speed applications [1].
Although the study of SNN has a very long history, its development has fallen far behind DNNs. Most of the studies are still limited to shallow networks rather than deep networks. Even for shallow SNNs, the training is still very difficult, the convergence is still very slow, and the testing performance is still much worse than DNNs. One of the commonly believed reasons is that SNNs use discrete spikes that are nondifferentiable. Nevertheless, although many techniques have been developed to approximate the nondifferentiable spiking pulses with smooth and differentiable pulses, the training and performance of SNNs are still not encouraging [2].
Another less-mentioned reason for the difficulty of training comes from the computational complexity perspective. Each SNN neuron's output is a time waveform rather than a single number as in the conventional DNN. Since the input-output response is described by a differential equation, many SNN training methods rely on solving such a differential equation online during training, which is computationally prohibitive in both the forward inference and the backward gradient backpropagation when the number of neurons becomes large.
Nevertheless, the third reason, which we believe more fundamental than the above two reasons but has not aroused too much attention, is that the SNN neurons adopted in most existing works have overly-complex and overly-nonlinear input-output responses that lead to slow and ill convergence during training and thus low performance during testing. Experience over DNNs has shown that while sufficient nonlinearity of the whole network is necessary, simplifying every single neuron's input-output nonlinearity can enhance both training speed and convergence [3]. Simplifying every single neuron's input-output expression also greatly reduces computational complexity and speeds up training, but this is secondary to the convergence issue.
Many different spiking neuron models have been used in SNNs, with leaky and nonleaky neurons, various spikes waveforms, and various kernel functions. There are a variety of different information coding methods, such as encoding information over spike rate, spike count, spike time, etc. Such variety is caused inherently by the variety of biological neurons. In general, neurons with higher bio-plausibility have more complex input-output expressions and are thus more difficult to train. SNNs should be developed as a trade-off between bio-plausibility and practicality.
In this paper, our first focus is to analyze the input-output expressions of the two general classes of SNN neurons, i.e., leaky and nonleaky neurons with single-spike temporal coding. We will show that leaky neurons are too complex to train while nonleaky neurons can build SNNs with easy training and high performance.
Our second focus is to demonstrate that SNNs built with nonleaky neurons can be superior to conventional DNNs once the training hurdle is overcome. We experiment with the SNNs over two popular network intrusion detection datasets: NSL-KDD and AWID [4], and show that they outperform a comprehensive list of existing DNN and classic machine learning algorithms.
The major contributions of this paper are listed as follows.
• We analyze the input-output response of two general types of SNN neurons to show that the commonly used leaky neurons have too complex and too nonlinear inputoutput responses and are thus hard to train. • We show that SNNs built with nonleaky neurons can have much less complex and much less nonlinear input-output response. They can be trained as easily as conventional DNNs. We provide a new training algorithm. • We train the proposed SNNs over two popular network intrusion detection datasets NSL-KDD and AWID. New benchmark results are obtained. To the best of our knowledge, this is the first time that SNN is reported for these two network intrusion detection datasets. This paper is organized as follows. Related works are introduced in Section II. The SNN neuron's input-output expressions are analyzed in Section III. The network intrusion datasets are described in Section IV. Experiments are presented in Section V and conclusions are given in Section VI.

II. RELATED WORK
SNNs use spike pattern flows to encode spatio-temporal information [5] [6]. The neurons can in general be classified into two classes: leaky neuron and nonleaky neuron. Most of the existing SNN works have focused on leaky neurons because it is considered as more bio-plausible.
SNN training methods can be categorized into three classes: unsupervised learning, supervised learning with indirect training, and supervised learning with direct training. For unsupervised learning, spike timing-dependent plasticity (STDP) is the most well-known one, which adjusts the weights connecting the pre-and post-synaptic neurons based on their relative spike times [6]- [9]. The dependency on the local neuronal activities without a global supervisor makes it have low performance.
For the second class, the most successful approach is to translate trained DNNs to SNNs [1] [10]. Zhang et al. [11] converted VGG16 to SNN with single-spike temporal coding. Although the translation approach can get the highest accuracy performance among SNNs so far, the loss of spike sparsity degrades energy efficiency.
For the third class, SpikeProp [12] minimized the loss between the true firing time and the single desired firing time with gradient descent rule over soft nonlinearity models. Gardner et al. [13] applied a probability neuron model to calculate gradients. With spike rate coding, gradients were calculated after impulse spikes were approximated by smooth functions in [14], [15]. Single-spike temporal coding was adopted in [16] where direct training was demonstrated in shallow networks over the MNIST dataset. Wu et al. [17] conducted direct training based on spatio-temporal back-propagation, where "spatio" referred to layer-by-layer gradient propagation and "temporal" referred to gradient propagation through timedomain spike waveform. Neftci et al. [2] used a recurrent neural network to calculate the time-domain spike waveform of nonleaky neurons.
Intrusion poses a serious risk in a network environment. Effective and efficient ways to detect the intrusion attacks and to enhance the network's ability to detect many types of intrusions have become important. Network intrusion detection algorithms can be categorized into three main groups: Statistics-based [18], knowledge-based [19] [20], and machine learning-based [21] [22]. The statistics-based approach involves collecting and examining every data record in a set of items and building a statistical model of normal user behavior. The knowledge-based approach tries to identify the requested actions from existing system data such as protocol specifications and network traffic instances. In contrast, the machine-learning method acquires complex pattern-matching capabilities from training data. Niyaz et al. studied deep learning over the NSL-KDD network intrusion detection dataset [23]. Shone et al. applied autoencoder [24] while Yin et al. applied a recurrent neural network to classify the NSL-KDD dataset [25]. Revyz et al. used deep learning to classify the AWID dataset [26]. Lopez-Martin et al. [4] compared the performance of reinforcement learning against a list of other machine learning methods over both the NSL-KDD and AWID datasets [4].

III. NEURON MODELS IN SPIKING NEURAL NETWORKS
In this section, we first analyze the input-output expressions of nonleaky and leaky integrate-and-fire spiking neurons. Then we develop SNN models based on the nonleaky neurons. Many SNNs use spike counts or rates to encode information, which may not be energy and time efficient enough because either a long time duration or a high rate count is needed for precise information estimation. Therefore, we consider neurons that use spiking time to encode information, and each neuron emits a single spike only for energy efficiency.
A. Input-output response of nonleaky spiking neuron For the non-leaky integrate-and-fire (n-LIF) neuron, if spiking time is used as information carrier, the membrane potential v j (t) of the neuron j is in general described by where w ji is the weight of the synaptic connection from the input neuron i to the output neuron j, t i is the spiking time of the input neuron i, g(t) is the synaptic current kernel function or spike waveform. The value of neuron i is encoded in the spike time t i . We assume g(t) = 0 for t < 0 or t > T . For single-spike neuron models, a neuron is allowed to spike only once unless the network is reset or a new input pattern is presented. Fig. 1(a) illustrates how this neuron works. During software implementation of the SNN or training SNN, we need to calculate t j from t i . We would better avoid using numerical ODE (ordinary differential equation) solver to calculate v j (t) and t j . Therefore, in this paper we focus on the ways of exploiting the solution of (1), i.e., to calculate t j . Let the spiking threshold voltage be v 0 . Then this neuron will spike at time From (3) we look for closed-form expressions for the output spiking time t j = f (t i ; w ji ). Closed-form expressions can be obtained in some special cases only, e.g., when G(t j − t i ) can be separated into G 1 (t j )G 2 (t i ) or G 1 (t j ) + G 2 (t i ). In the sequel, we consider two such special cases. a) Case 1. Unit-step spike: Let To simplify notation, we assume that T > t j , which means the neuron spike waveform is a unit step function. Then In this case, from (3) we can find where the set C = {∀k : t k < t j } includes all the input neurons (and only these neurons) that have spike time t k less than the output spike time t j . The above equation can be written as the standard DNN neuron's input-output expression form (weight plus bias) as The composite weights wji ∈C w j and bias v0/a ∈C w j are nonlinear. There is no other nonlinear activation function. One can add extra nonlinear activation to t j , but our experiments indicate that this is not necessary. Since (7) is in a similar form to conventional DNN input-output expression, gradient back-propagation and training can be conducted in a similar way as DNN.
b) Case 2. Exponentially-decaying spike: In this case, the spike waveform is where τ is the decaying time constant. Then we have Considering the spiking threshold v 0 , the neuron j's spike time can be calculated from In software implementation of SNN, we can simply use e ti/τ and e tj /τ as the input and output neuron values. The input-output response (11) is similar to the DNN neuron's input-output response. There is no bias term, and we do not need other nonlinear activation because the composite weights

B. Input-output response of leaky spiking neuron
For the leaky integrate-and-fire (LIF) neuron, the membrane potential of the neuron j is modeled as where b is a positive constant representing the decaying rate of the membrane potential when there is no input. Fig. 1 illustrates how this neuron model work. Note the decaying of membrane potential after t 1 .
To solve (12), let µ(t) = e bdt = e bt , and multiply µ(t) to both sides of (12). We obtain which can be written as Integrating both sides, we can get which can be deducted as Assume zero-initial condition, i.e., v j (0) = 0, and exploit the fact that g(t) = 0 for t < 0. We can get Closed-form solutions to t j based on (17) can only be obtained under some special cases. Note that if b = 0, it reduces back to the nonleaky neuron case. a) Case 1. Unit-step spike: In this case, let g(t) = a for 0 ≤ t ≤ T and 0 otherwise. Assume T > t j , from (17) Rearranging the terms, we arrive at which is similar to the conventional DNN input-output expressions if we let the values of the input neuron and the output neuron be e bti and e btj , respectively. Nonlinearity is embedded in the composite weights. Interestingly, this case is similar to the nonleaky with exponentially-decaying spike case, as (19) is similar to (11). b) Case 2. Exponentially-decaying spike: In this case, the spike waveform is the same as (8). From (17) As pointed out in [27], there are only two special parameter settings that we can find closed-form solution to t j . The first parameter setting is bτ = 1 while the second parameter setting is bτ = 1/2. Obviously, such settings severely limit their practical applicability. For the first parameter setting bτ = 1, (20) becomes At v j (t j ) = v 0 , rearranging the terms of (21) we obtain whose solution can be expressed as the Lambert W function as where t j is the correct output and t is calculated as a function of first weight w j1 . Loss function is 0 at the true weight. 5 input neurons. t i and w ji are randomly generated, v 0 = 1, b = 0.5, τ = 1, α = 1. Complex (nonreal) Lambert W function and square-root values in (23)(25) We can find the solution as It can been seen that e Nevertheless, in both of the above two parameter settings, the computational complexity becomes prohibitive when the number of neurons becomes big. More importantly, each neuron's input-output becomes overly-complex and overlynonlinear, which is harmful to the training of deep neural networks. Fig. 2 shows some typical t j ∼ t i curves and cost function (t−t j ) 2 curves for the above 5 cases, from which we can see that the leaky neuron can have very complex nonlinear patterns.
Although the above deduction is just for two special parameter settings, we expect that such overly-nonlinear property holds in general for leaky neurons and argue that this is one of the key reasons that SNNs build with leaky neurons are difficult to train and have low performance.

C. SNNs Built with n-LIF Neurons
From the above analysis we can see that it is better to adopt nonleaky neurons when building SNNs. Biological neurons are Algorithm 1 Forward pass of a SNN neuron based on (11) Input: z = [z 1 , · · · , z N ]: input spiking time vector Input: w = [w 1 , · · · , w N ]: weight vector Output: z out : output spiking time i ← argsort(z): ascending order index [1]: index of the first True element return z out ← z candidate [k]: output spiking time in general leaky. With periodic resetting back to zero initial state, the nonleaky neuron model can in fact approximate well the biological leaky neuron. Between the unit-step spike and the exponentially-decaying spike, we adopt the latter because it is more bio-plausible and more energy efficient. Therefore, (11) is used to model neuron input-output response for our SNNs.
For software implementation, we skip the details of v j (t) and just use (11) to calculate neuron values. We also let z j = αe tj /τ with a constant α as the neuron value. We need to sort the input z i = αe ti/τ from small to large so as to determine the set C, which leads to extra computational complexity. Note that hardware implementation does not suffer this problem because t i is the arrival timing of input spikes which are automatically sorted.
We developed Algorithm 1 to calculate (11). This algorithm can be applied to all the cases studied in the previous subsection, with appropriate z i and expressions to calculate z j . For this we just need to define neuron values as n-LIF, Case-1 αe ti/τ , n-LIF, Case-2 αe bti , LIF, Case-1 αt i , LIF, Case-2, 1st Setting αe ti/2τ , LIF, Case-2, 2nd Setting (26) for the various cases analyzed in the previous subsection. We also need to replace the ways of calculating z cumsum , w cumsum and z candidate of Algorithm 1 with the corresponding equations (7), (11), (19), (23) and (25).
The SNN model based on (11) was initially proposed in [16]. Nevertheless, only some shallow networks with 2 ∼ 3 fully-connected layers were proposed and experimented over the simple XOR and MNIST dataset with mediocre performance. Compared with our Algorithm 1, the algorithm presented in [16] does not force t j > t i for every i ∈ C, and thus does not match well with hardware implementation. It uses "for" loop rather than the "cumsum" function. In addition, it requires ∈C w j > v 0 /τ while we do not. This last requirement might lead to slow and worse training convergence because weights are forced to be positive biased. Since all the neuron values z i are positive, this may lead to an SNN with mostly positive weights and neuron values which may not perform well.
For an L-layer SNN, define the input as z 0 with elements z 0,i = αe t0,i/τ and the final output as z L with elements z L,i = αe t L,i /τ . Then we have z L = f (z 0 ; w) with nonlinear mapping f and trainable weight w which includes all weights w ji . Let the targeting output be class c. We train the network with the loss function The first term is to make z L,c the smallest (equivalently t L,c the smallest) one. With time-to-first-spike (TTFS) encoding, the classification result is made at the time of the first spike among output neurons. Smaller z L,c or e tj means stronger classification output. The second term is L 2 regularization to prevent weights from becoming too large. The third term (with β ≥ 1) is the weight sum cost, which enlarges each neuron's input weight summation to increase its firing probability. The parameters K and λ are weighting coefficients. Based on the loss function (27) and Algorithm 1 for z calculation, we can implement the training optimization conveniently in the conventional DNN training platforms. Thanks to the closed-form expression for the neuron spiking time (11), gradient calculation is easy and gradient-based backpropagation can be used to train the weights for our SNNs. The training becomes nothing different from conventional DNNs.
In contrast, if we build SNNs with LIF neurons, except using the undesired unit-step spiking waveform, we do not have such easy training because the closed-form solutions to t j are too complex. Complex neuron input-output leads to the highly nonlinear response for each neuron, which makes the training hard to converge. In addition, we just have two special parameter settings where the closed-form solutions exist for LIF neurons, and these two special parameter settings may not fit well with practical neuron hardware circuits. Except for these two special settings, the general LIF neurons do not even have closed-form solutions exist, which makes the computational complexity extremely high.

IV. NETWORK INTRUSION DETECTION DATASETS A. NSL-KDD Dataset
The earliest effort to create an intrusion detection dataset was made by DARPA (Defense Advanced Research Project Agency) in 1998. They created the KDD98 (Knowledge Discovery and Data Mining (KDD)) dataset, which was the basis for the KDD Cup99 dataset. The KDD Cup99 dataset contained a wide variety of intrusions simulated in a military network environment. It consists of approximately 4,900,000 data records, each of which is a vector of extracted feature values from a connection record obtained from the raw network data gathered during the simulated intrusions. A connection is a sequence of TCP packets from some IP addresses. Each connection is labeled as either normal or one specific kind of attack. The simulated attacks fall into one of the following four categories: 1) DOS -Denial of Service (e.g. a syn flood), 2) R2L -Unauthorized access from a remote machine (e.g. password guessing), 3) U2R -Unauthorized access to superuser or root functions (e.g. a buffer overflow attack), 4) Probing -Surveillance and other probing for vulnerabilities (e.g. port scanning). Each data record consists of 41 features, most of which take on continuous values. The features include the basic features of an individual TCP connection such as duration, protocol type, number of bytes transferred, and the flag indicating the normal or error status of the connection. Some other features of individual connections are obtained using domain knowledge, such as the number of file creation operations, the number of failed login attempts, whether root shell was obtained, and others. There are also a number of features computed using a two-second time window, such as the number of connections to the same host as the current connections within the past two seconds, percent of connections that have "SYN" and "REJ" errors, and the number of connections to the same service as the current connection within the past two seconds.
The NSL-KDD dataset was developed from the KDD Cup99 dataset [28] after a statistical analysis performed on the KDD Cup99 dataset raised important issues that heavily influenced the intrusion detection accuracy and resulted in a misleading evaluation. Specifically, one of the main problems in the KDD Cup99 dataset is the huge amount of duplicated packets. Tavallaee et al. analyzed the KDD training and test datasets and revealed that approximately 78% and 75% of the network packets are duplicated. This huge quantity of duplicated instances in the training set would influence machinelearning methods to be biased towards normal instances and thus prevent them from learning irregular instances which are typically more damaging. The NSL-KDD dataset was built to resolve these problems by eliminating the duplicated records. The NSL-KDD train dataset consists of 125,973 records and the test dataset contains 22,544 records. There are 22 training intrusion attacks and 41 features.
In this paper, we used the KDDTrain+, KDDTest+ and KDDTest21 sets of the NSL-KDD dataset. The KDDTrain+ set contains a total of 125,973 instances comprising of 58,630 instances of attack traffics and 67,343 instances of normal traffics. The KDDTest+ set contains a total of 22,544 instances. As a subset of the KDDTest+ set, the KDDTest21 set includes a total of 11,850 instances. Cross-validation was conducted over the KDDTrain+ set in our experiments. We also considered validation using the simple hold-out (train-test) approach applied on the KDDTest+ and KDDTest-21 sets.
In order to compare fairly with [4] and the results listed over there, we first used the identical datasets included with the source code of [4]. We did the same processing to extend the feature of the NSL-KDD dataset from 41 to 122. We call this the "Original" dataset.
Next, to further enhance classification performance, we preprocessed the NSL-KDD dataset with a method similar to a Gaussian Receptive Field data to spike conversion, which extended the number of features of the NSL-KDD dataset from 41 to 312. We call this the "Resampled" dataset.

B. Aegean WiFI Intrusion Dataset (AWID)
The AWID was published in 2015 as a collection of WiFi network data consisting of real traces of both normal and intrusion data collected from real network environments [29]. Each record in the dataset is represented as a vector of 155 attributes, and each attribute has numeric or nominal values. Based on the number of target classes, the dataset can be classified into the AWID-CLS dataset and the AWIDATK dataset. The AWID-CLS dataset groups the instances into 4 main classes including normal, flooding, impersonation, and injection. The AWID-ATK dataset has 17 detailed target classes that belong to these 4 main classes. Based on the number of instances, each of these two datasets has two different versions: Full Set and Reduced Set. It is important to mention that these two versions are not related. The reduced set was collected independently from the full set at different times, with different tools, and in different environments.
In this paper, we conducted experiments with the reducedset version of the four-class dataset (AWID-CLS-R-Tst). The AWID-CLS-R-Tst set includes a total of 575,643 instances. To compare fairly with the results listed in [4], first, we used the "Original" dataset with 46 features which were obtained from the data included in the source code of [4]. Next, to enhance performance, we preprocessed the dataset to extend the number of features from 46 to 206, which we call "Resampled" dataset.

A. Network Models Used in Experiments
For the NSL-KDD "Original" dataset, we designed an SNN with three fully-connected layers: the first layer has 100 neurons, the second layer has 100 neurons, and the last layer has 5 neurons. The input is a batch vector of dimension 122. For the NSL-KDD "Resampled" dataset, we applied this same SNN architecture, but the input is a batch vector of dimension 312.
We compared our SNN's performance with the results listed in [4], which is fair because we used the same training and testing datasets as the latter. Nevertheless, the state-of-the-art performance listed in [4] was relatively low. As pointed out over there, much higher performance was claimed in some other literature. Unfortunately, these results claimed with high performance was not reproducible due to lack of implementation details, lack of source code, and unknown testing dataset. Such literature was omitted in [4]. Even though we also doubt these high claims, we believe that the DNN's performance may not be limited by the out-dated cases used in [4]. Therefore, we designed and trained our own DNNs, specifically, a DNN model with fully connected layers and a CNN model with 1D convolutional layers. This permits us to compare SNN more fairly with the state-of-the-art DNNs under similar experiment settings. Our DNN model has three fully-connected layers with 100, 100, and 5 neurons respectively. The architecture of the CNN is: Conv1D (312,100), Conv1D (312,100), Dropoutlayer, Dense-layer (5). It has two 1D convolutional layers. Each layer has data dimension 312 and feature map dimension 100. The filter kernel size is 3.
For the AWID "Original" dataset, we designed an SNN with three fully-connected layers: the first layer has 100 neurons, the second layer has 100 neurons, and the last layer has 4 neurons. The input is a batch vector of dimension 46. For the AWID "Resampled" dataset, we applied this same SNN, but the input is a batch vector of dimension 206.
Similarly, we also designed our own DNN model with fully connected layers and our own CNN model with 1D convolutional layers. This permits us to compare SNN with DNN more fairly. The DNN model has three fully-connected layers with 100, 100, and 4 neurons respectively. The architecture of the CNN model is: Conv1D (206,100), Conv1D (206,100), Dropout-layer, Dense-layer (4). The filter kernel size is 3.
To train the SNN, we applied hyper-parameters K = 100, λ = 0.001, Adam optimizer with learning rate 0.001 for the "Original" dataset and 10 −5 for the "Resampled" dataset, and batch size 128. We applied t i = 2d i and α = 1 to map the input data d i to input neuron spiking time. We assumed τ = 1 and v 0 = 1 for the neurons. Through experiments, we found that keeping the training running for a lot of more iterations at a small learning rate can effectively increase classification accuracy.

B. Experiment Results over NSK-KDD Dataset
The classification results are shown in Table I. We used the following four evaluation metrics: classification accuracy, F1, precision, and recall. Note that except for our models, i.e., our DNN, our CNN-1D, SNN (Original), SNN (Resampled), the results of all other models were obtained from [4]. From the table, we can easily see that our CNN-1D model had classification accuracy 0.9564 and outperformed all the results listed in [4]. More importantly, both of our SNN models outperformed all the other models, including all the DNN-based models. Our SNN achieved accuracy 0.9717 with the original dataset, and achieved 0.9931 accuracy with the resampled dataset. This showed that SNN can be competitive to DNNs. This also showed that appropriate data preprocessing can further enhance performance. Table II shows the classification performance for each of the 5 classes. We can see that our SNN model can classify each sample with extremely high accuracy.

C. Experiment Results over AWID Dataset
The classification results are shown in Table III. Except for our 4 models, the results of all the other models were obtained from [4]. We can see that both our DNN and our CNN-1D models achieved competitive performance as existing

VI. CONCLUSIONS
In this paper we develop single-spike temporal-coded SNNs that can be easily trained with competitive performance as conventional DNNs. We analyzed systematically the inputoutput expressions of single-spike temporal-coded leaky and nonleaky neurons. We show that the commonly used leaky neurons have overly-complex and overly-nonlinear responses and argue that this is the primary reason that makes SNNs hard to train and low in performance. We also show that with nonleaky neurons we can resolve these problems. We demonstrate this by experimenting with such SNNs over the two popular network intrusion detection datasets and by showing that the SNNs outperformed a list of existing methods including the DNN-based methods. The easy training and high performance indicate that SNNs can be competitive to DNNs and are promising for practical applications.

Supplementary Material
VII. EXTRA EXPERIMENT RESULTS: NSL-KDD DATASET Distribution of the five classes (normal + 4 attack types) of the NSL-KDD dataset is shown in Fig. 3. For the NSL-KDD "Original" dataset, Table V lists the total number of data records and the number of data records classified correctly for each class. Fig. 4 shows the classification confusion matrix expressed as the number of data records tested. Table VI shows the classification performance for each of the 5 classes and Fig. 5 shows the confusion matrix expressed as a percentage. Fig. 6 shows the classification results for each class.   For the NSL-KDD "Resampled" dataset, Table VII lists the total number of data records and the number of data records classified correctly for each class. Fig. 7 shows the classification confusion matrix expressed as the number of data records tested. Table II shows the classification performance for each of the 5 classes and Fig. 8 shows the confusion matrix expressed as percentages. Fig. 9 shows the classification results for each class.     The distribution of 4 classes (normal + 3 attack classes) of the AWID dataset is shown in Fig. 10. For the AWID "Original" dataset, Table VIII lists the total number of data records and the number of data records classified correctly for each class. Fig. 11 shows the classification confusion matrix expressed as the number of data records tested. Table IX shows the classification performance for each of the 5 classes and Fig. 12 shows the confusion matrix expressed as percentages. Fig. 13 shows the classification results for each class. For the AWID "Resampled" dataset, Table X lists the total number of data records and the number of data records classified correctly for each class. Fig. 14 shows the classification confusion matrix expressed as the number of data records tested. Table IV shows the classification performance for each of the 5 classes and Fig. 15 shows the confusion matrix   expressed as percentages. Fig. 16 shows the classification results for each class.  Table XI and Table  XII. Its performance over the AWID dataset on each attack class is listed in Table XIII and Table XIV.    For our CNN-1D model, its performance over the NSL-KDD dataset on each attack class is listed in Table XV and  Table XVI. Its performance over the AWID dataset on each attack class is listed in Table XVII and Table XVIII.