Biologically Motivated Algorithms for Propagating Local Target
  Representations
  Finding biologically plausible alternatives to back-propagation of errors is
a fundamentally important challenge in artificial neural network research. In
this paper, we propose a learning algorithm called error-driven Local
Representation Alignment (LRA-E), which has strong connections to predictive
coding, a theory that offers a mechanistic way of describing neurocomputational
machinery. In addition, we propose an improved variant of Difference Target
Propagation, another procedure that comes from the same family of algorithms as
LRA-E. We compare our procedures to several other biologically-motivated
algorithms, including two feedback alignment algorithms and Equilibrium
Propagation. In two benchmarks, we find that both of our proposed algorithms
yield stable performance and strong generalization compared to other competing
back-propagation alternatives when training deeper, highly nonlinear networks,
with LRA-E performing the best overall.
