Credit Assignment in Neural Networks through Deep Feedback Control
  The success of deep learning sparked interest in whether the brain learns by
using similar techniques for assigning credit to each synaptic weight for its
contribution to the network output. However, the majority of current attempts
at biologically-plausible learning methods are either non-local in time,
require highly specific connectivity motives, or have no clear link to any
known mathematical optimization method. Here, we introduce Deep Feedback
Control (DFC), a new learning method that uses a feedback controller to drive a
deep neural network to match a desired output target and whose control signal
can be used for credit assignment. The resulting learning rule is fully local
in space and time and approximates Gauss-Newton optimization for a wide range
of feedback connectivity patterns. To further underline its biological
plausibility, we relate DFC to a multi-compartment model of cortical pyramidal
neurons with a local voltage-dependent synaptic plasticity rule, consistent
with recent theories of dendritic processing. By combining dynamical system
theory with mathematical optimization theory, we provide a strong theoretical
foundation for DFC that we corroborate with detailed results on toy experiments
and standard computer-vision benchmarks.
