Simple Neural Network Can't Learn Xor

Neural networks XOR returns incorrect output

You have a small implementation error :

in the Backpropagation, you evaluate :

hidden_errors = output_weights.T.dot(output_errors)

but your hidden error must be evaluate based on the d_predicted_output, like so :

hidden_errors = output_weights.T.dot(d_predicted_output)

Also, you should decrease your learning rate and increase to number of epochs. 10000 epochs and lr = 0.1 works for me, but you can fine tune this.

XOR not learned using keras v2.0

I cannot add a comment to Daniel's response as I don't have enough reputation, but I believe he's on the right track. While I have not personally tried running the XOR with Keras, here's an article that might be interesting - it analyzes the various regions of local minima for a 2-2-1 network, showing that higher numerical precision would lead to fewer instances of getting stuck on a gradient descent algorithm.

The Local Minima of the Error Surface of the 2-2-1 XOR Network (Ida G. Sprinkhuizen-Kuyper and Egbert J.W. Boers)

On a side note I won't consider using a 2-4-1 network as over-fitting the problem. Having 4 linear cuts on the 0-1 plane (cutting into a 2x2 grid) instead of 2 cuts (cutting the corners off diagonally) just separates the data in a different way, but since we only have 4 data points and no noise in the data, the neural network that uses 4 linear cuts isn't describing "noise" instead of the XOR relationship.

Neural Network Back-Propagation Algorithm Gets Stuck on XOR Training PAttern

LiKao's comment to simplify my implementation and get rid of the object-oriented aspects solved my problem. The flaw in the algorithm as it is described above is unknown, however I now have a working neural network that is a lot smaller.

Feel free to continue to provide insight on the problem with my previous implementation, as others may have the same problem in the future.

How do I train a neural network capable of calculating XOR using MATLAB?

You need more training examples. Repeat some of them.

In my case I tried:

net = feedforwardnet([3]);
%Repeat 4 times
P = [0 0; 0 1; 1 0; 1 1; 0 0; 0 1; 1 0; 1 1; 0 0; 0 1; 1 0; 1 1; 0 0; 0 1; 1 0; 1 1]';
T = [0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0]; % desired output
net = configure(net, P, T);
net.trainParam.goal = 1e-8;
net.trainParam.epochs = 1000;
net = train(net, P, T);

And the result is

 sim(net, P)

ans =
  Columns 1 through 8
    0.0000    1.0000    1.0000    0.0000    0.0000    1.0000    1.0000    0.0000
  Columns 9 through 16
    0.0000    1.0000    1.0000    0.0000    0.0000    1.0000    1.0000    0.0000

---

Related Topics