Neural networks embrace learned diversity
  Diversity conveys advantages in nature, yet homogeneous neurons typically
comprise the layers of artificial neural networks. Here we construct neural
networks from neurons that learn their own activation functions, quickly
diversify, and subsequently outperform their homogeneous counterparts.
Sub-networks instantiate the neurons, which meta-learn especially efficient
sets of nonlinear responses. Such learned diversity provides examples of
dynamical systems selecting diversity over uniformity and elucidates the role
of diversity in natural and artificial systems.
