Online Memorization of Random Firing Sequences by a Recurrent Neural
  Network
  This paper studies the capability of a recurrent neural network model to
memorize random dynamical firing patterns by a simple local learning rule. Two
modes of learning/memorization are considered: The first mode is strictly
online, with a single pass through the data, while the second mode uses
multiple passes through the data. In both modes, the learning is strictly local
(quasi-Hebbian): At any given time step, only the weights between the neurons
firing (or supposed to be firing) at the previous time step and those firing
(or supposed to be firing) at the present time step are modified. The main
result of the paper is an upper bound on the probability that the single-pass
memorization is not perfect. It follows that the memorization capacity in this
mode asymptotically scales like that of the classical Hopfield model (which, in
contrast, memorizes static patterns). However, multiple-rounds memorization is
shown to achieve a higher capacity (with a nonvanishing number of bits per
connection/synapse). These mathematical findings may be helpful for
understanding the functions of short-term memory and long-term memory in
neuroscience.
