Deep Reservoir Networks with Learned Hidden Reservoir Weights using
  Direct Feedback Alignment
  Deep Reservoir Computing has emerged as a new paradigm for deep learning,
which is based around the reservoir computing principle of maintaining random
pools of neurons combined with hierarchical deep learning. The reservoir
paradigm reflects and respects the high degree of recurrence in biological
brains, and the role that neuronal dynamics play in learning. However, one
issue hampering deep reservoir network development is that one cannot
backpropagate through the reservoir layers. Recent deep reservoir architectures
do not learn hidden or hierarchical representations in the same manner as deep
artificial neural networks, but rather concatenate all hidden reservoirs
together to perform traditional regression. Here we present a novel Deep
Reservoir Network for time series prediction and classification that learns
through the non-differentiable hidden reservoir layers using a
biologically-inspired backpropagation alternative called Direct Feedback
Alignment, which resembles global dopamine signal broadcasting in the brain. We
demonstrate its efficacy on two real world multidimensional time series
datasets.
