Deep Learning with Darwin: Evolutionary Synthesis of Deep Neural
  Networks
  Taking inspiration from biological evolution, we explore the idea of "Can
deep neural networks evolve naturally over successive generations into highly
efficient deep neural networks?" by introducing the notion of synthesizing new
highly efficient, yet powerful deep neural networks over successive generations
via an evolutionary process from ancestor deep neural networks. The
architectural traits of ancestor deep neural networks are encoded using
synaptic probability models, which can be viewed as the `DNA' of these
networks. New descendant networks with differing network architectures are
synthesized based on these synaptic probability models from the ancestor
networks and computational environmental factor models, in a random manner to
mimic heredity, natural selection, and random mutation. These offspring
networks are then trained into fully functional networks, like one would train
a newborn, and have more efficient, more diverse network architectures than
their ancestor networks, while achieving powerful modeling capabilities.
Experimental results for the task of visual saliency demonstrated that the
synthesized `evolved' offspring networks can achieve state-of-the-art
performance while having network architectures that are significantly more
efficient (with a staggering $\sim$48-fold decrease in synapses by the fourth
generation) compared to the original ancestor network.
