On the Modeling of Error Functions as High Dimensional Landscapes for
  Weight Initialization in Learning Networks
  Next generation deep neural networks for classification hosted on embedded
platforms will rely on fast, efficient, and accurate learning algorithms.
Initialization of weights in learning networks has a great impact on the
classification accuracy. In this paper we focus on deriving good initial
weights by modeling the error function of a deep neural network as a
high-dimensional landscape. We observe that due to the inherent complexity in
its algebraic structure, such an error function may conform to general results
of the statistics of large systems. To this end we apply some results from
Random Matrix Theory to analyse these functions. We model the error function in
terms of a Hamiltonian in N-dimensions and derive some theoretical results
about its general behavior. These results are further used to make better
initial guesses of weights for the learning algorithm.
