A New Training Method for Feedforward Neural Networks Based on Geometric   Contraction Property of Activation Functions
  We propose a new training method for a feedforward neural network having the activation functions with the geometric contraction property. The method consists of constructing a new functional that is less nonlinear in comparison with the classical functional by removing the nonlinearity of the activation function from the output layer. We validate this new method by a series of experiments that show an improved learning speed and better classification error. 
Petre Birtea, Cosmin Cernazanu-Glavan, Alexandru Sisu
cs.NE cs.LG