Article
Posted 24 Jun 2007
Stats
741.5K views 44.2K downloads 282 bookmarked

Comments and Discussions


Hi,
The context:
I have a device which returns me a vector of data when a rfid tag passes nearby. The vector looks like the following:
[30, 45, 60, 50, 31, 0, 264, 641, 999, 1310, 32, 40, 55, 51, 33, 725, 200, 600, 900, 1200]
I would like to train the network with many tags, which means many similar vectors with little variance. When the training is over, I want to pass a test tag in front of the device, obtain a vector of data, and get from the network if the data somehow "fits" to the trained data.
For a better understanding, the data in the vector changes if the tag goes leftright or updown. I would train using the same "direction" and test with a tag in the other direction, leading to a "rejection" of the item.
My question is :
Does it make sense to use a neural network (even yours) for such a case?
My training data might be composed of 5000 vectors. The way I see how I could use your application library is to define each of these vectors as "pattern", and then look at the probabilities if the test vector reaches for example >60% similarities with one of the vectors.
Could you help me with this?
Thanks a lot,
JF
JeanFrancois Dufour





Hi,
if the inputs are not linearly separable, neural networks should be used.
to use NNs for this purpose modify "NeuralNetwork.cs" as:

public bool Train()
{
double currentError = 0;
int currentIteration = 0;
NeuralEventArgs Args = new NeuralEventArgs() ;
do
{
currentError = 0;
foreach (double[] p in TrainingSet)//##################
{
NeuralNet.ForwardPropagate(p, "anything");//just one output that is useless
NeuralNet.BackPropagate();
currentError += NeuralNet.GetError();
}
currentIteration++;
if (IterationChanged != null && currentIteration % 5 == 0)
{
Args.CurrentError = currentError;
Args.CurrentIteration = currentIteration;
IterationChanged(this, Args);
}
} while (currentError > maximumError && currentIteration < maximumIteration && !Args.Stop);
if (IterationChanged != null)
{
Args.CurrentError = currentError;
Args.CurrentIteration = currentIteration;
IterationChanged(this, Args);
}
if (currentIteration >= maximumIteration  Args.Stop)
return false;//Training Not Successful
return true;
}

before creating NeuralNetwork:
define:
ArrayList TrainingSet=new ArrayList();
//add 5000 double[] to ArrayList
//of course double arrays must contain values between 01
//a vector [30, 45, 60, 50, ..., 600, 900, 1200] is not valid
//but dividing all elements to max number is valid (here divide all values to 1200)
NN must contain 1 output and be created as("preinputnum" is the number of elements in a vector):
neuralNetwork = new NeuralNetwork<string>(new BP3Layer<string>(preinputnum, InputNum, HiddenNum, 1), TrainingSet);
then pass any double[] to this function(T is string):
public void Recognize(double[] Input, ref T MatchedHigh, ref double OutputValueHight, ref T MatchedLow, ref double OutputValueLow)
"OutputValueHight" will retrieve the similarity.
hope this helps.
Murat





This is by far the easiest Neural Network article to understand. Thank you for the crystal clear code.






In NeuralDemo.cs should HiddenNum be:
} else if (comboBoxLayers.SelectedIndex == 2) {
int InputNum = Int16.Parse(textBoxInputUnit.Text);
// int HiddenNum = Int16.Parse(textBoxOutputUnit.Text);
int HiddenNum = Int16.Parse(textBoxHiddenUnit.Text);
if (TrainingSet != null) {
Charles





It has been a late reply but now I fixed it in the current version.
Thanks.
Murat





Hi, your source code is easy to understand and compile. Thanks.
However I think there is an error in the Backpropagation function.
It is missing a factor of "Output*(1Output) for all hidden layers.
The link below gives a very good explanation of the math and easy to follow.
http://www.speech.sri.com/people/anand/771/html/node37.html
I am trying to use the structure of your code to do some real work and can't afford to have any error in the algorithm.
It would be great if you could clarify if I am wrong.
Thanks.
Lee





You are right. I removed Output*(1Output) factor from hidden layers. Since, unfortunately, it causes trainig process an "endless process". Maybe it stems from selecting wrong initial values, or wrong choice of hidden layer number or anything else. But, I think, if the purpose of traing network is to decrease the error, this also works fine(I can not prove ) .If you want to apply original theory do that:
private void BackPropagate()<br />
{<br />
int i, j;<br />
double total;<br />
<br />
//Fix Hidden Layer's Error<br />
for (i = 0; i < HiddenNum; i++)<br />
{<br />
total = 0.0;<br />
for (j = 0; j < OutputNum; j++)<br />
{<br />
total += HiddenLayer[i].Weights[j] * OutputLayer[j].Error;<br />
}<br />
//HiddenLayer[i].Error = total;<br />
HiddenLayer[i].Error = HiddenLayer[i].Error * (1  HiddenLayer[i].Error) * total;<br />
}<br />
<br />
//Fix Input Layer's Error<br />
for (i = 0; i < InputNum; i++)<br />
{<br />
total = 0.0;<br />
for (j = 0; j < HiddenNum; j++)<br />
{<br />
total += InputLayer[i].Weights[j] * HiddenLayer[j].Error;<br />
}<br />
//InputLayer[i].Error = total;<br />
InputLayer[i].Error = InputLayer[i].Error * (1  InputLayer[i].Error) * total;<br />
<br />
.... <br />
}<br />





Hi, thanks for the clarification. Though I think there is an error where you used "Error" instead of "Output" for the hidden layer/InputLayer. Should it have been the following?
//Fix Hidden Layer's Error
for (i = 0; i < HiddenNum; i++)
{
total = 0.0;
for (j = 0; j < OutputNum; j++)
{
total += HiddenLayer[i].Weights[j] * OutputLayer[j].Error;
}
//HiddenLayer[i].Error = total;
HiddenLayer[i].Error = HiddenLayer[i].Output * (1  HiddenLayer[i].Output) * total;
}
//Fix Input Layer's Error
for (i = 0; i < InputNum; i++)
{
total = 0.0;
for (j = 0; j < HiddenNum; j++)
{
total += InputLayer[i].Weights[j] * HiddenLayer[j].Error;
}
//InputLayer[i].Error = total;
InputLayer[i].Error = InputLayer[i].Output * (1  InputLayer[i].Output) * total;





disadvantage of copy and paste..
the form of
HiddenLayer[i].Error = HiddenLayer[i].Output * (1  HiddenLayer[i].Output) * total;<br />
InputLayer[i].Error = InputLayer[i].Output * (1  InputLayer[i].Output) * total;
is right





Hello ,
I am a student studying in Computer Science
Your product is so wonderful , impressing me the most
In fact , while selfstudying the neural network ...
I find it somewhat difficult to understand both the number of hidden layers and the number of hidden layers ' node used in neural network
If more hidden layers , will it be better in result ?
For instance , the input is 48 X 48 pixel images
How many input layer nodes , hidden layer nodes ,ouput layer nodes it should be ?
In your application , is it only restricted to 0/1 in value of input or output ? can I use this to identify something colorful ?
If not I remember wrong , some application may have threshold value , implying that it seems to turn out 0/1
result . If I want to apply this on continious values ,
do I need to implement the threshould ?
By the way , Is there any simple neural network tutorial
website recommended , I want to learn more .... >_<"
Thanks You Very Much
Your Product is worthy of full mark ( 100 )





Actually, selecting proper numbers for hidden layer's nodes is confusing. there are some theoritical expressions about it but, nevertheless, not so clear.
As I said at previous comment, if there is an input having 600 nodes, first hidden sholud have 250 nodes, second hidden should have 100 nodes and output have 34 units([from input to output] node numbers should be choosen in descending order).
I dont use 01 as input but double. the following method, for example, converts a bitmap(which is input) to double array:
<br />
private double[] IMGToDoubleArr(Bitmap BM, int Col, int Row) <br />
{<br />
double HRate = ((Double)Row / BM.Height);<br />
double WRate = ((Double)Col / BM.Width);<br />
double[] Result = new double[Col * Row];<br />
for (int r = 0; r < Row; r++)<br />
{<br />
for (int c = 0; c < Col; c++)<br />
{<br />
Color color = BM.GetPixel((int)(c / WRate), (int)(r / HRate));<br />
Result[r * Col + c] =1 (color.R * .3 + color.G * .59 + color.B * .11) / 255;<br />
}<br />
}<br />
return Result;<br />
}
To make more clear, I didnt use threshold values(as not required to use). Finally, I uploaded some material on neural networks as I am asked so frequently. here is the link:
h**p://rapidshare.com/files/55104312/Books.rar.html





Dear Mr. Murat
I want to ask you about your aplication "C# Implementation of BackPropagation Neural Network For Pattern Recognition".
can you explain me more details about the tab settings.I don't understand what for is "Number of layers", "Maximum Error", "Number of Input Unit", "Number of Hidden Unit", why default value for "Number of Input Unit" is 250, and why default value for "Number of Hidden Unit" is 100.
Thank you very much for your attention.





A neural network structure contains 1 or more layers. the first layer is called "input layer" and the last one is called "output layer". Layers between the first and last layer called "hidden layers".
A layer contains many nodes. If input layer contains 600 nodes, then we say number of input unit is 600. Dont keep your mind on default values of layer units. they are just "default" and can be changed. but if network contains 600 inputs and 34 outputs, hidden layers (which are between input and output layer) should have unit numbers between 600 and 34. in this example, input have 600, first hidden have 250, second hidden have 100 and output have 34 units.
Finally, "Maximum error" indicates how well the network will be trained.(a smaller value of "Maximum error" means, network will be trained better)





Hello,
I am a student from Croatia who studies Computer science. I have to write cca. 20 pages about: Use of neural networks for pattern recognition.
describe: types of networks which are used in this purpose
commercial tools
achieved results
I found your work on http://www.codeproject.com/ so I thought that you could help me beacuse you are probably good in this field.
Any help will be useful... Links, pdfs, texts ... about my task.
Can You contact me on shecccATgmail.com ?
Thank You very much!





If I have an image with a string of characters like ABC123, will this work on that image also? Do I need to normalize and wash out the image to make the letters appear black and white before this will work??





An image containing ABC123 should be divided into images as A,B,D,1,2,3 to be recognized. It is also possible to train the network to recognize images containing 6 letters but probably no any PC would train such a network. (As only (28+10) output is enought for the first case, however, there must be 38*38*38*38*38*38 outputs for the second case.) Probably, it would take a simple PC days to train such a network
It does'nt required to map pixels to 0 and 1 or wash out. Gray level pixels also work fine.





I am not one to comment normally, but good job, I have been timidly browsing the net for something like this a couple of days now, exactly what I was looking for. Could use a couple more comments =], I am just sitting around here looking at your code, and I was wondering, what do I need to change to have the program check just say the border of an image? Again, Great Work, thanks!
Jeff





Thanks.
To Check the Borders of Images:
1Insert the pixels [in the borders of images] into TrainingSet as array of double.
2Train Network
3Send the borders of an image [as the parameter] to Recognize method.





Great work.
Only problem with the source code is that you are trying to update the UI from a secondary thread which causes an error but I fixed this using invoke on textBoxState with no problems.





Thanks much for notifying. I fixed it in the current version.





is there any current version of this application?






General News Suggestion Question Bug Answer Joke Praise Rant Admin Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

