Integration of Leaky-Integrate-and-Fire-Neurons in Deep Learning
  Architectures
  Up to now, modern Machine Learning is mainly based on fitting high
dimensional functions to enormous data sets, taking advantage of huge hardware
resources. We show that biologically inspired neuron models such as the
Leaky-Integrate-and-Fire (LIF) neurons provide novel and efficient ways of
information encoding. They can be integrated in Machine Learning models, and
are a potential target to improve Machine Learning performance.
  Thus, we derived simple update-rules for the LIF units from the differential
equations, which are easy to numerically integrate. We apply a novel approach
to train the LIF units supervisedly via backpropagation, by assigning a
constant value to the derivative of the neuron activation function exclusively
for the backpropagation step. This simple mathematical trick helps to
distribute the error between the neurons of the pre-connected layer. We apply
our method to the IRIS blossoms image data set and show that the training
technique can be used to train LIF neurons on image classification tasks.
Furthermore, we show how to integrate our method in the KERAS (tensorflow)
framework and efficiently run it on GPUs. To generate a deeper understanding of
the mechanisms during training we developed interactive illustrations, which we
provide online.
  With this study we want to contribute to the current efforts to enhance
Machine Intelligence by integrating principles from biology.
