Neocortical plasticity: an unsupervised cake but no free lunch
  The fields of artificial intelligence and neuroscience have a long history of
fertile bi-directional interactions. On the one hand, important inspiration for
the development of artificial intelligence systems has come from the study of
natural systems of intelligence, the mammalian neocortex in particular. On the
other, important inspiration for models and theories of the brain have emerged
from artificial intelligence research. A central question at the intersection
of these two areas is concerned with the processes by which neocortex learns,
and the extent to which they are analogous to the back-propagation training
algorithm of deep networks. Matching the data efficiency, transfer and
generalization properties of neocortical learning remains an area of active
research in the field of deep learning. Recent advances in our understanding of
neuronal, synaptic and dendritic physiology of the neocortex suggest new
approaches for unsupervised representation learning, perhaps through a new
class of objective functions, which could act alongside or in lieu of
back-propagation. Such local learning rules have implicit rather than explicit
objectives with respect to the training data, facilitating domain adaptation
and generalization. Incorporating them into deep networks for representation
learning could better leverage unlabelled datasets to offer significant
improvements in data efficiency of downstream supervised readout learning, and
reduce susceptibility to adversarial perturbations, at the cost of a more
restricted domain of applicability.
