15,947,389 members
See more:
So I am trying to implement an MLP for XOR. Currently I am not getting any satisfying results. I coded it in java and currently I'm convinced that the backpropagation is correct and so is the forward but I don't get why it doesn't work. The delta calculation is what I had the mot problem with but I think I understand it now. I hope someone could help me. I have set the learning rate from 0.1 to 0.7 and no luck.
I already tested with the inputs keeping it simple I used 2 inputs: 00, 01,10,11. The results I'm getting:
0.03046590405786709
0.9159156257800326
0.8664182057539402
0.7590399051848435

They should be 0110 but the network isn't learning properly. I tested the results from the forward and they were correct so the only one I'm not quite sure how to check are the back propagation results. The delta results seem to be okay as well. Please guide me if my algorithm understanding is wrong.

What I have tried:

Java
```public double[] forward(ArrayList<Double> inputs){
// Loop through the hidden layers
for(int i = 0; i < NH; i++){
Hidden[i] = 0.0;
for(int j = 0; j < NI; j++){
Hidden[i] += (input[j] * W1[j][i]);
}

Hidden[i] +=bias;
//Calculate activation
Hidden[i] = 1/ (1 + Math.exp(-Hidden[i]));

}

// Outer layer - output calculation
for(int i = 0; i < NO; i++){
Output[i] = 0.0;
for(int j = 0; j < NH; j++){
Output[i] += (Hidden[j] * W2[j][i]);
}

Output[i] +=bias;
//Calculate activation
Output[i] = 1/ (1 + Math.exp(-Output[i]));
}
return Output;
}```

Java
```public double backwards(ArrayList<Double> targetOutputs){
double error = 0;

for(int i = 0; i < NO; i++){
Z2[i] = Output[i]*(1.0-Output[i]) * (targetOutputs.get(i) - Output[i]);

}

for(int i=0; i< NH; i++) {
error = 0.0;
for(int j=0; j< NO; j++){
error += W2[i][j] * Z2[j];
}
Z1[i] = Hidden[i] * (1.0-Hidden[i]) * error;

}
//calculate the delta weight change
for(int i=0; i< NH; i++) {
for(int j=0; j< NI; j++) {
dW1[j][i] += Z1[i] * input[j] + LR*dW1[j][i];

}
}
for(int i=0; i< NO; i++){
for(int j=0; j< NH; j++){
dW2[j][i] +=  Z2[i] * Hidden[j] + LR*dW2[j][i];
}
}
return Z2[0];
}```

Java
```public void updateWeights(double LR){
for(int i= 0; i< NO; i++){
for(int j=0; j< NH; j++){
W2[j][i] += dW2[j][i] ;
dW2[j][i] = 0;
}
}
for(int i=0; i< NH; i++){
for(int j=0; j< NI; j++){
W1[j][i] +=  dW1[j][i];
dW1[j][i] = 0;
}
}
}
```
Posted
Updated 2-May-16 6:04am
v3
Patrice T 2-May-16 11:36am
What is MLP ? My Little Pony ?
Member 12497969 2-May-16 11:42am
Better(Title)?
Patrice T 2-May-16 11:44am
sure :)

## Solution 1

Quote:
I'm convinced that the backpropagation is correct and so is the forward
being convinced is not enough ! you must ensure that things are correct.
Try unit testing, either with real unit testing or manually: get a sample dataset, run the function on it, check that the results match the expectations.

It is also a good idea to use the debugger to see where something strange append.
The debugger allow you to follow the execution line by line, inspect variables and you will see that there is a point where it stop doing what you expect.
Debugger - Wikipedia, the free encyclopedia[^]
Mastering Debugging in Visual Studio 2010 - A Beginner's Guide[^]
http://docs.oracle.com/javase/7/docs/technotes/tools/windows/jdb.html[^]
https://www.jetbrains.com/idea/help/debugging-your-first-java-application.html[^]

Member 12497969 2-May-16 11:50am
I alrady tested with the inputs keeping it simple I used 2 inputs: 00, 01,10,11. The results I'm getting:
0.03046590405786709
0.9159156257800326
0.8664182057539402
0.7590399051848435

They should be 0110 but the network isn't learning properly. I tested the results from the forward and they were correct so the only one Im not quite sure how to check are the back propagation results. The delta results seem to be ok as well. I dont usually ask questions online and this is my first time. Please guide me if my algorithm understanding is wrong.
Patrice T 2-May-16 11:59am
Use Improve question to update your question.
Member 12497969 4-May-16 10:00am
Thanks, it turns out that the weights were not randomised properly.
Patrice T 4-May-16 10:04am
Nice to see you solved the problem;
did you used the debugger ?