Tourbillon: a Physically Plausible Neural Architecture
  In a physical neural system, backpropagation is faced with a number of
obstacles including: the need for labeled data, the violation of the locality
learning principle, the need for symmetric connections, and the lack of
modularity. Tourbillon is a new architecture that addresses all these
limitations. At its core, it consists of a stack of circular autoencoders
followed by an output layer. The circular autoencoders are trained in
self-supervised mode by recirculation algorithms and the top layer in
supervised mode by stochastic gradient descent, with the option of propagating
error information through the entire stack using non-symmetric connections.
While the Tourbillon architecture is meant primarily to address physical
constraints, and not to improve current engineering applications of deep
learning, we demonstrate its viability on standard benchmark datasets including
MNIST, Fashion MNIST, and CIFAR10. We show that Tourbillon can achieve
comparable performance to models trained with backpropagation and outperform
models that are trained with other physically plausible algorithms, such as
feedback alignment.
