Approximating Real-Time Recurrent Learning with Random Kronecker Factors
  Despite all the impressive advances of recurrent neural networks, sequential
data is still in need of better modelling. Truncated backpropagation through
time (TBPTT), the learning algorithm most widely used in practice, suffers from
the truncation bias, which drastically limits its ability to learn long-term
dependencies.The Real-Time Recurrent Learning algorithm (RTRL) addresses this
issue, but its high computational requirements make it infeasible in practice.
The Unbiased Online Recurrent Optimization algorithm (UORO) approximates RTRL
with a smaller runtime and memory cost, but with the disadvantage of obtaining
noisy gradients that also limit its practical applicability. In this paper we
propose the Kronecker Factored RTRL (KF-RTRL) algorithm that uses a Kronecker
product decomposition to approximate the gradients for a large class of RNNs.
We show that KF-RTRL is an unbiased and memory efficient online learning
algorithm. Our theoretical analysis shows that, under reasonable assumptions,
the noise introduced by our algorithm is not only stable over time but also
asymptotically much smaller than the one of the UORO algorithm. We also confirm
these theoretical results experimentally. Further, we show empirically that the
KF-RTRL algorithm captures long-term dependencies and almost matches the
performance of TBPTT on real world tasks by training Recurrent Highway Networks
on a synthetic string memorization task and on the Penn TreeBank task,
respectively. These results indicate that RTRL based approaches might be a
promising future alternative to TBPTT.
