Alternating optimization method based on nonnegative matrix
  factorizations for deep neural networks
  The backpropagation algorithm for calculating gradients has been widely used
in computation of weights for deep neural networks (DNNs). This method requires
derivatives of objective functions and has some difficulties finding
appropriate parameters such as learning rate. In this paper, we propose a novel
approach for computing weight matrices of fully-connected DNNs by using two
types of semi-nonnegative matrix factorizations (semi-NMFs). In this method,
optimization processes are performed by calculating weight matrices
alternately, and backpropagation (BP) is not used. We also present a method to
calculate stacked autoencoder using a NMF. The output results of the
autoencoder are used as pre-training data for DNNs. The experimental results
show that our method using three types of NMFs attains similar error rates to
the conventional DNNs with BP.
