Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function:
  Learning with Backpropagation
  The timing of individual neuronal spikes is essential for biological brains
to make fast responses to sensory stimuli. However, conventional artificial
neural networks lack the intrinsic temporal coding ability present in
biological networks. We propose a spiking neural network model that encodes
information in the relative timing of individual neuron spikes. In
classification tasks, the output of the network is indicated by the first
neuron to spike in the output layer. This temporal coding scheme allows the
supervised training of the network with backpropagation, using locally exact
derivatives of the postsynaptic spike times with respect to presynaptic spike
times. The network operates using a biologically-plausible alpha synaptic
transfer function. Additionally, we use trainable synchronisation pulses that
provide bias, add flexibility during training and exploit the decay part of the
alpha function. We show that such networks can be trained successfully on noisy
Boolean logic tasks and on the MNIST dataset encoded in time. The results show
that the spiking neural network outperforms comparable spiking models on MNIST
and achieves similar quality to fully connected conventional networks with the
same architecture. We also find that the spiking network spontaneously
discovers two operating regimes, mirroring the accuracy-speed trade-off
observed in human decision-making: a slow regime, where a decision is taken
after all hidden neurons have spiked and the accuracy is very high, and a fast
regime, where a decision is taken very fast but the accuracy is lower. These
results demonstrate the computational power of spiking networks with biological
characteristics that encode information in the timing of individual neurons. By
studying temporal coding in spiking networks, we aim to create building blocks
towards energy-efficient and more complex biologically-inspired neural
architectures.
