
Comments and Discussions



The key feature of neural network is its threshold value
but I wonder why your network didn't have this ....?





Threshold value is used at the early models of neural networks. Backpropagation uses sigmoid instead of threshold..





As I know, in your program the number of input units of network is determined by the number of pixels of pattern. But I found that the patterns in your file do not have the same size. There are 23x33, 29x33, 19x33... I would like to know when I train the network, how many input units are there indeed? It is impossible to train the same network if the patterns have different dimension.
Thank you for your response.





[All] images are scaled to average height and width of [all] images. Then inserted to TrainingSet and the NN is trained..





Hi everybody!
My name is Khiem , I'm from Vietnam.
I downloaded the demo project including the source code 5 days ago for research but I got a problem installing the software in my PC. I do not install the C# program for my PC yet , do I need to install it?
The fact that I used my colleges computer to download it , and this computer doesn't have C# yet , ironically , it works .
So please tell me what steps I should do to make it work . Thanks so much for your help .
Khiem





Demo does not have any installer. Just click the *.exe file to run it [at a computer that .NET Framework 2.0 is installed]





:DI really think you work is really great.
I have some questions about the settings.
1. Is "Number of Layers" means that in addition to input, there are "Number of Layers" layers in the network? For example, if it is 1, the network consists one input and 1 output layer.
2.Number of Input Unit: why is it only for two or three layer net?
3.Number of Input Unit: Should it be the number of pixels of pattern? Why is its default value 295. It should be 27x33 that many.
4. What is the activation function f in this program?
Thank you very much. I appreciate your response. Thank you.
randy





1. Exactly
2. Number of Input Unit is actually 1st hidden layer so it does not exist for a NN that has 1 layer
3. numofpreinput is the number of pixels at an image. (27x33)
4. activation function is sigmoit:
public double F(double x)
{
return (1 / (1 + Math.Exp(x)));
}





Greetings,
First of all i would to thank you, for doing such a good work and giving to public....
When i was going through your work, i was stuck at one point, you have used
InputLayerNode = (networkInput + NumOfPatterns) * .33
HiddenLayerNode = (networkInput + NumOfPatterns) * .11
Is there any specific reason behind using this formula for input and hidden node..
Do we have any generic formula for Input and hidden Node..
I am new buddy in this field. so plz guide me...
Waiting for ur reply...
Thanking you....
Vimal





nothing special. just hidden layers' node number should have the values between input and output layers'
cheers..





Hi
u have provided a very good neural network system.
But i did not understand the logic of how did u decide the number of input units, hidden units and output units. and how does the output units determine which image it is liek suppose u gave 34 output units then how do those 34 units wil decide whch image of training set it is
pls reply as soon as possible





if the bitmap is 20x30 then there are 600 inputs (each pixel is mapped to one input). and if there are 34 training pattern there are 34 outputs (each pattern is mapped to one output, say "0" refers to 1st output, "1" refers to 2nd output.. "z" refers to 34th output..) hidden unit number may differ between number of input and output units.





Hi,
The context:
I have a device which returns me a vector of data when a rfid tag passes nearby. The vector looks like the following:
[30, 45, 60, 50, 31, 0, 264, 641, 999, 1310, 32, 40, 55, 51, 33, 725, 200, 600, 900, 1200]
I would like to train the network with many tags, which means many similar vectors with little variance. When the training is over, I want to pass a test tag in front of the device, obtain a vector of data, and get from the network if the data somehow "fits" to the trained data.
For a better understanding, the data in the vector changes if the tag goes leftright or updown. I would train using the same "direction" and test with a tag in the other direction, leading to a "rejection" of the item.
My question is :
Does it make sense to use a neural network (even yours) for such a case?
My training data might be composed of 5000 vectors. The way I see how I could use your application library is to define each of these vectors as "pattern", and then look at the probabilities if the test vector reaches for example >60% similarities with one of the vectors.
Could you help me with this?
Thanks a lot,
JF
JeanFrancois Dufour





Hi,
if the inputs are not linearly separable, neural networks should be used.
to use NNs for this purpose modify "NeuralNetwork.cs" as:

public bool Train()
{
double currentError = 0;
int currentIteration = 0;
NeuralEventArgs Args = new NeuralEventArgs() ;
do
{
currentError = 0;
foreach (double[] p in TrainingSet)//##################
{
NeuralNet.ForwardPropagate(p, "anything");//just one output that is useless
NeuralNet.BackPropagate();
currentError += NeuralNet.GetError();
}
currentIteration++;
if (IterationChanged != null && currentIteration % 5 == 0)
{
Args.CurrentError = currentError;
Args.CurrentIteration = currentIteration;
IterationChanged(this, Args);
}
} while (currentError > maximumError && currentIteration < maximumIteration && !Args.Stop);
if (IterationChanged != null)
{
Args.CurrentError = currentError;
Args.CurrentIteration = currentIteration;
IterationChanged(this, Args);
}
if (currentIteration >= maximumIteration  Args.Stop)
return false;//Training Not Successful
return true;
}

before creating NeuralNetwork:
define:
ArrayList TrainingSet=new ArrayList();
//add 5000 double[] to ArrayList
//of course double arrays must contain values between 01
//a vector [30, 45, 60, 50, ..., 600, 900, 1200] is not valid
//but dividing all elements to max number is valid (here divide all values to 1200)
NN must contain 1 output and be created as("preinputnum" is the number of elements in a vector):
neuralNetwork = new NeuralNetwork(new BP3Layer(preinputnum, InputNum, HiddenNum, 1), TrainingSet);
then pass any double[] to this function(T is string):
public void Recognize(double[] Input, ref T MatchedHigh, ref double OutputValueHight, ref T MatchedLow, ref double OutputValueLow)
"OutputValueHight" will retrieve the similarity.
hope this helps.
Murat





This is by far the easiest Neural Network article to understand. Thank you for the crystal clear code.






In NeuralDemo.cs should HiddenNum be:
} else if (comboBoxLayers.SelectedIndex == 2) {
int InputNum = Int16.Parse(textBoxInputUnit.Text);
// int HiddenNum = Int16.Parse(textBoxOutputUnit.Text);
int HiddenNum = Int16.Parse(textBoxHiddenUnit.Text);
if (TrainingSet != null) {
Charles





It has been a late reply but now I fixed it in the current version.
Thanks.
Murat





Hi, your source code is easy to understand and compile. Thanks.
However I think there is an error in the Backpropagation function.
It is missing a factor of "Output*(1Output) for all hidden layers.
The link below gives a very good explanation of the math and easy to follow.
http://www.speech.sri.com/people/anand/771/html/node37.html
I am trying to use the structure of your code to do some real work and can't afford to have any error in the algorithm.
It would be great if you could clarify if I am wrong.
Thanks.
Lee





You are right. I removed Output*(1Output) factor from hidden layers. Since, unfortunately, it causes trainig process an "endless process". Maybe it stems from selecting wrong initial values, or wrong choice of hidden layer number or anything else. But, I think, if the purpose of traing network is to decrease the error, this also works fine(I can not prove ) .If you want to apply original theory do that:
private void BackPropagate()
{
int i, j;
double total;
//Fix Hidden Layer's Error
for (i = 0; i < HiddenNum; i++)
{
total = 0.0;
for (j = 0; j < OutputNum; j++)
{
total += HiddenLayer[i].Weights[j] * OutputLayer[j].Error;
}
//HiddenLayer[i].Error = total;
HiddenLayer[i].Error = HiddenLayer[i].Error * (1  HiddenLayer[i].Error) * total;
}
//Fix Input Layer's Error
for (i = 0; i < InputNum; i++)
{
total = 0.0;
for (j = 0; j < HiddenNum; j++)
{
total += InputLayer[i].Weights[j] * HiddenLayer[j].Error;
}
//InputLayer[i].Error = total;
InputLayer[i].Error = InputLayer[i].Error * (1  InputLayer[i].Error) * total;
....
}





Hi, thanks for the clarification. Though I think there is an error where you used "Error" instead of "Output" for the hidden layer/InputLayer. Should it have been the following?
//Fix Hidden Layer's Error
for (i = 0; i < HiddenNum; i++)
{
total = 0.0;
for (j = 0; j < OutputNum; j++)
{
total += HiddenLayer[i].Weights[j] * OutputLayer[j].Error;
}
//HiddenLayer[i].Error = total;
HiddenLayer[i].Error = HiddenLayer[i].Output * (1  HiddenLayer[i].Output) * total;
}
//Fix Input Layer's Error
for (i = 0; i < InputNum; i++)
{
total = 0.0;
for (j = 0; j < HiddenNum; j++)
{
total += InputLayer[i].Weights[j] * HiddenLayer[j].Error;
}
//InputLayer[i].Error = total;
InputLayer[i].Error = InputLayer[i].Output * (1  InputLayer[i].Output) * total;





disadvantage of copy and paste..
the form of
HiddenLayer[i].Error = HiddenLayer[i].Output * (1  HiddenLayer[i].Output) * total;
InputLayer[i].Error = InputLayer[i].Output * (1  InputLayer[i].Output) * total;
is right





Hello ,
I am a student studying in Computer Science
Your product is so wonderful , impressing me the most
In fact , while selfstudying the neural network ...
I find it somewhat difficult to understand both the number of hidden layers and the number of hidden layers ' node used in neural network
If more hidden layers , will it be better in result ?
For instance , the input is 48 X 48 pixel images
How many input layer nodes , hidden layer nodes ,ouput layer nodes it should be ?
In your application , is it only restricted to 0/1 in value of input or output ? can I use this to identify something colorful ?
If not I remember wrong , some application may have threshold value , implying that it seems to turn out 0/1
result . If I want to apply this on continious values ,
do I need to implement the threshould ?
By the way , Is there any simple neural network tutorial
website recommended , I want to learn more .... >_<"
Thanks You Very Much
Your Product is worthy of full mark ( 100 )





Actually, selecting proper numbers for hidden layer's nodes is confusing. there are some theoritical expressions about it but, nevertheless, not so clear.
As I said at previous comment, if there is an input having 600 nodes, first hidden sholud have 250 nodes, second hidden should have 100 nodes and output have 34 units([from input to output] node numbers should be choosen in descending order).
I dont use 01 as input but double. the following method, for example, converts a bitmap(which is input) to double array:
private double[] IMGToDoubleArr(Bitmap BM, int Col, int Row)
{
double HRate = ((Double)Row / BM.Height);
double WRate = ((Double)Col / BM.Width);
double[] Result = new double[Col * Row];
for (int r = 0; r < Row; r++)
{
for (int c = 0; c < Col; c++)
{
Color color = BM.GetPixel((int)(c / WRate), (int)(r / HRate));
Result[r * Col + c] =1 (color.R * .3 + color.G * .59 + color.B * .11) / 255;
}
}
return Result;
}
To make more clear, I didnt use threshold values(as not required to use). Finally, I uploaded some material on neural networks as I am asked so frequently. here is the link:
h**p://rapidshare.com/files/55104312/Books.rar.html





Dear Mr. Murat
I want to ask you about your aplication "C# Implementation of BackPropagation Neural Network For Pattern Recognition".
can you explain me more details about the tab settings.I don't understand what for is "Number of layers", "Maximum Error", "Number of Input Unit", "Number of Hidden Unit", why default value for "Number of Input Unit" is 250, and why default value for "Number of Hidden Unit" is 100.
Thank you very much for your attention.







General News Suggestion Question Bug Answer Joke Rant Admin Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Type  Article 
Licence  CPOL 
First Posted  24 Jun 2007 
Views  426,019 
Downloads  27,004 
Bookmarked  256 times 

