Interpretable Deep Recurrent Neural Networks via Unfolding Reweighted
  $\ell_1$-$\ell_1$ Minimization: Architecture Design and Generalization
  Analysis
  Deep unfolding methods---for example, the learned iterative shrinkage
thresholding algorithm (LISTA)---design deep neural networks as learned
variations of optimization methods. These networks have been shown to achieve
faster convergence and higher accuracy than the original optimization methods.
In this line of research, this paper develops a novel deep recurrent neural
network (coined reweighted-RNN) by the unfolding of a reweighted
$\ell_1$-$\ell_1$ minimization algorithm and applies it to the task of
sequential signal reconstruction. To the best of our knowledge, this is the
first deep unfolding method that explores reweighted minimization. Due to the
underlying reweighted minimization model, our RNN has a different
soft-thresholding function (alias, different activation functions) for each
hidden unit in each layer. Furthermore, it has higher network expressivity than
existing deep unfolding RNN models due to the over-parameterizing weights.
Importantly, we establish theoretical generalization error bounds for the
proposed reweighted-RNN model by means of Rademacher complexity. The bounds
reveal that the parameterization of the proposed reweighted-RNN ensures good
generalization. We apply the proposed reweighted-RNN to the problem of video
frame reconstruction from low-dimensional measurements, that is, sequential
frame reconstruction. The experimental results on the moving MNIST dataset
demonstrate that the proposed deep reweighted-RNN significantly outperforms
existing RNN models.
