Evolutionary Synthesis of Deep Neural Networks via Synaptic
  Cluster-driven Genetic Encoding
  There has been significant recent interest towards achieving highly efficient
deep neural network architectures. A promising paradigm for achieving this is
the concept of evolutionary deep intelligence, which attempts to mimic
biological evolution processes to synthesize highly-efficient deep neural
networks over successive generations. An important aspect of evolutionary deep
intelligence is the genetic encoding scheme used to mimic heredity, which can
have a significant impact on the quality of offspring deep neural networks.
Motivated by the neurobiological phenomenon of synaptic clustering, we
introduce a new genetic encoding scheme where synaptic probability is driven
towards the formation of a highly sparse set of synaptic clusters. Experimental
results for the task of image classification demonstrated that the synthesized
offspring networks using this synaptic cluster-driven genetic encoding scheme
can achieve state-of-the-art performance while having network architectures
that are not only significantly more efficient (with a ~125-fold decrease in
synapses for MNIST) compared to the original ancestor network, but also
tailored for GPU-accelerated machine learning applications.
