Predictive Coding: Towards a Future of Deep Learning beyond
  Backpropagation?
  The backpropagation of error algorithm used to train deep neural networks has
been fundamental to the successes of deep learning. However, it requires
sequential backward updates and non-local computations, which make it
challenging to parallelize at scale and is unlike how learning works in the
brain. Neuroscience-inspired learning algorithms, however, such as
\emph{predictive coding}, which utilize local learning, have the potential to
overcome these limitations and advance beyond current deep learning
technologies. While predictive coding originated in theoretical neuroscience as
a model of information processing in the cortex, recent work has developed the
idea into a general-purpose algorithm able to train neural networks using only
local computations. In this survey, we review works that have contributed to
this perspective and demonstrate the close theoretical connections between
predictive coding and backpropagation, as well as works that highlight the
multiple advantages of using predictive coding models over
backpropagation-trained neural networks. Specifically, we show the
substantially greater flexibility of predictive coding networks against
equivalent deep neural networks, which can function as classifiers, generators,
and associative memories simultaneously, and can be defined on arbitrary graph
topologies. Finally, we review direct benchmarks of predictive coding networks
on machine learning classification tasks, as well as its close connections to
control theory and applications in robotics.
