Single neuron-based neural networks are as efficient as dense deep
  neural networks in binary and multi-class recognition problems
  Recent advances in neuroscience have revealed many principles about neural
processing. In particular, many biological systems were found to
reconfigure/recruit single neurons to generate multiple kinds of decisions.
Such findings have the potential to advance our understanding of the design and
optimization process of artificial neural networks. Previous work demonstrated
that dense neural networks are needed to shape complex decision surfaces
required for AI-level recognition tasks. We investigate the ability to model
high dimensional recognition problems using single or several neurons networks
that are relatively easier to train. By employing three datasets, we test the
use of a population of single neuron networks in performing multi-class
recognition tasks. Surprisingly, we find that sparse networks can be as
efficient as dense networks in both binary and multi-class tasks. Moreover,
single neuron networks demonstrate superior performance in binary
classification scheme and competing results when combined for multi-class
recognition.
