Supervised level-wise pretraining for recurrent neural network
  initialization in multi-class classification
  Recurrent Neural Networks (RNNs) can be seriously impacted by the initial
parameters assignment, which may result in poor generalization performances on
new unseen data. With the objective to tackle this crucial issue, in the
context of RNN based classification, we propose a new supervised layer-wise
pretraining strategy to initialize network parameters. The proposed approach
leverages a data-aware strategy that sets up a taxonomy of classification
problems automatically derived by the model behavior. To the best of our
knowledge, despite the great interest in RNN-based classification, this is the
first data-aware strategy dealing with the initialization of such models. The
proposed strategy has been tested on four benchmarks coming from two different
domains, i.e., Speech Recognition and Remote Sensing. Results underline the
significance of our approach and point out that data-aware strategies
positively support the initialization of Recurrent Neural Network based
classification models.
