I've implemented a neural network and I'm training it to compute Xor. 1 out of x times it fails to learn, where x is about 5 or 10. It then gives e.g. 0.67 instead of 0 as output for input (1,1). Is this just some unlucky randomization of the initial weights and should I move on to my real problem instance, or should I solve this first? What could be the cause?

Some more background info:

I'm using f(x) = 1/(1+exp(-x)) as activation function for both hidden neurons and output neuron. The hidden and output neuron have a bias. All weights are initially random numbers between 0 and 1. I'm using the backpropagation algorithm as described here: https://en.wikipedia.org/wiki/Backpropagation

I varied the learning factor from 0.001 to 1 and I did up to 1,000,000 training iterations.

  • $\begingroup$ I think it is possible if you are starting from very bad starting conditions. It also depends on the structure of the network and the data used to train it. $\endgroup$ – Donbeo May 20 '16 at 17:41


There are 16 local minimums that have the highest conversion if the weights are initialized between 0.5 and 1.

enter image description here

Image source: (Hirose, Yamashita, and Huiya 1991)

| cite | improve this answer | |

Your Answer

