Self-organization of multi-layer spiking neural networks
  Living neural networks in our brains autonomously self-organize into large,
complex architectures during early development to result in an organized and
functional organic computational device. A key mechanism that enables the
formation of complex architecture in the developing brain is the emergence of
traveling spatio-temporal waves of neuronal activity across the growing brain.
Inspired by this strategy, we attempt to efficiently self-organize large neural
networks with an arbitrary number of layers into a wide variety of
architectures. To achieve this, we propose a modular tool-kit in the form of a
dynamical system that can be seamlessly stacked to assemble multi-layer neural
networks. The dynamical system encapsulates the dynamics of spiking units,
their inter/intra layer interactions as well as the plasticity rules that
control the flow of information between layers. The key features of our
tool-kit are (1) autonomous spatio-temporal waves across multiple layers
triggered by activity in the preceding layer and (2) Spike-timing dependent
plasticity (STDP) learning rules that update the inter-layer connectivity based
on wave activity in the connecting layers. Our framework leads to the
self-organization of a wide variety of architectures, ranging from multi-layer
perceptrons to autoencoders. We also demonstrate that emergent waves can
self-organize spiking network architecture to perform unsupervised learning,
and networks can be coupled with a linear classifier to perform classification
on classic image datasets like MNIST. Broadly, our work shows that a dynamical
systems framework for learning can be used to self-organize large computational
devices.
