Ghost Units Yield Biologically Plausible Backprop in Deep Neural
  Networks
  In the past few years, deep learning has transformed artificial intelligence
research and led to impressive performance in various difficult tasks. However,
it is still unclear how the brain can perform credit assignment across many
areas as efficiently as backpropagation does in deep neural networks. In this
paper, we introduce a model that relies on a new role for a neuronal inhibitory
machinery, referred to as ghost units. By cancelling the feedback coming from
the upper layer when no target signal is provided to the top layer, the ghost
units enables the network to backpropagate errors and do efficient credit
assignment in deep structures. While considering one-compartment neurons and
requiring very few biological assumptions, it is able to approximate the error
gradient and achieve good performance on classification tasks. Error
backpropagation occurs through the recurrent dynamics of the network and thanks
to biologically plausible local learning rules. In particular, it does not
require separate feedforward and feedback circuits. Different mechanisms for
cancelling the feedback were studied, ranging from complete duplication of the
connectivity by long term processes to online replication of the feedback
activity. This reduced system combines the essential elements to have a working
biologically abstracted analogue of backpropagation with a simple formulation
and proofs of the associated results. Therefore, this model is a step towards
understanding how learning and memory are implemented in cortical multilayer
structures, but it also raises interesting perspectives for neuromorphic
hardware.
