Backprojection for Training Feedforward Neural Networks in the Input and
  Feature Spaces
  After the tremendous development of neural networks trained by
backpropagation, it is a good time to develop other algorithms for training
neural networks to gain more insights into networks. In this paper, we propose
a new algorithm for training feedforward neural networks which is fairly faster
than backpropagation. This method is based on projection and reconstruction
where, at every layer, the projected data and reconstructed labels are forced
to be similar and the weights are tuned accordingly layer by layer. The
proposed algorithm can be used for both input and feature spaces, named as
backprojection and kernel backprojection, respectively. This algorithm gives an
insight to networks with a projection-based perspective. The experiments on
synthetic datasets show the effectiveness of the proposed method.
