Neural Network Classifier

Sample Image - Demo.png

Introduction

Neural Network is a powerful tool used in modern intelligent systems. Nowadays, many applications that involve pattern recognition, feature mapping, clustering, classification and etc. use Neural Networks as an essential component. In recent decades, several types of neural networks have been developed. Back Error Propagation, Kohonen feature map and Hopfield network are some of basic networks that have been developed and are used in many applications.

In this article general Multi Layer Perceptron or Back Error Propagation is presented. The sample application uses this Multi Layer Perceptron and classifies the blue and red patterns generated by user.

Network structure

Figure below shows a typical MLP with three layers of weights.

Sample screenshot

In a typical neural network, these objects exist: layers (usually 3 layers), some neurons in each layer and synapses between layers. Based on these objects, I developed appropriate classes.

Classes used in MLP

Neuron: This is the basic element of network, that has some members such as output, sum and delta.

   static double momentum;// = 0.9;

   static double learningRate;// = 0.05;

   CList<Synapse*> inlinks;//a list of synapses brings to this neuron

   CList<Synapse*> outlinks;
   double output; // range from 0.0 to 1.0

   double sum; // weighted sum of inputs

   double delta;
   CString label;

Layer: This class has a list of neurons and some methods for computing errors and weight adjustment.

   CList<Neuron*> neurons;
   int size;
   CString name;
   Layer(CString strLbl,int size);
   Neuron* getNeuron(int i);
   void computeOutputs();
   void computeBackpropDeltas(Samples s); // for output neurons

   void computeBackpropDeltas(); // for hidden neurons

   void computeWeights();

Synapse: This class has its weight and two pointers to its connected neurons (from and to).

   double weight;//weight between "to" and "from" neurons

   double data;
   Neuron* from;
   Neuron* to;

Perceptron: This is the last class that has some lists of layers, inputSamples and outputSamples (targets).

   CList<Layer*> layers;
   CList<Samples> inputSamples;//Samples are array of double

   CList<Samples> outputSamples;
   Layer* inputLayer;
   Layer* outputLayer;
   double error;

How to use Sample Application

To use this program follow these steps:

Clear screen by pressing on the Clear button.
Define your network structure by putting appropriate values in H1, H2 and H3 edit boxes. These are the number of neurons in hidden layers. If some value was 0, that layer will not be included in network (by default the network has three layers: one input, one hidden and one output).
Put some blue and red points on screen by clicking on it. (Select color from combo box).
Now you can press �initialize� network button.
Set maximum number of iterations and press �learn� as more as the network correctly classifies all points on screen. You can see the error graph by clicking on �Show error graph�. The �smartgraph� component is used in this project and you must download and register it on your computer to see error graph correctly. It is available here.
Error values can be written into a text file, if you check in �Write errors to file�.

This program is fully object oriented and can be understood easily if you are familiar with Multi Layer Perceptron. At last, I used this class to develop a Farsi OCR application and it was working very well!.

Use and enjoy�

You must Sign In to use this message board.


Per page

help me
lechat2326

8:19 28 Sep '09

Please, would you like to help me to prepare my project handwritten recognition word or letters with TDNN (time delay neuronal network) ,a language is c# or . Thank you very match Contact me in: Doc_dh23000@hotmail.com
Sign In·View Thread·PermaLink

Red&Blue vs Yellow&Blue, see similar software: Sharky Neural Network
SharkTime

8:44 6 Aug '09

Great. I wrote similar software: Sharky Neural Network
Try and compare with yours.

SNN is a free software for playing with neural networks classification.
You can control more parameters.

I got some problems using your software, suddenly your application closed during learning.

You may consider remove 'initialize network' feature, and add it by default to Learn button.
If it is not good for you, you can make two buttons: Learn (with initialization), and Learn more (without init.).
It will be much easier to use it, just one click to start.
Also, more points at start is better idea. Neural networks likes more learning data.

Rgds,
SharkTime.com

image processing & neural network
Member 3521894

22:22 29 Jun '09

سلام
من در زمینه پردازش تصویر و شبکه عصبی در حال تحقیق و پیاده سازی پروژه ای هستم و پروژه مورد نظرم را با مطلب پیاده سازی کرده ام و الان می خواهم معادل توابع مطلب را در ویژوال
استودیو پیدا کنم و
این کار را با استفاده از کتابخانه های
open cv و emgu
انجام می دهم.
و چون آشنایی کافی با این کتابخانه ها ندارم نتوانسته ام این کار را انجام دهم. لطفا اگر در این زمینه کار کرده اید من را راهنمایی بفرمایید.
در کل من می خواهم شبکه عصبی ای که در مطلب پیاده سازی کرده ام را در ویژوال استودیو (در سی شارپ) پیاده سازی کنم

help text to speech
elhamhaj

20:08 31 Jan '09

proje man dar morede text to speeche with neural network. age mitoonid komakam konid.
Sign In·View Thread·PermaLink	2.00/5

SOM
aparna tiwari

3:31 18 Jan '09

hi i want to work on a SOM subject for my major project. of cource i already worked on MLP please help me
Sign In·View Thread·PermaLink	1.00/5

Exe crashes under Vista
DPLNeural

0:45 1 Oct '08

has anyone experienced problems running this code in Vista? I compiled the project under VS 2008 but it still crashed...any similar experiences as it ran fine under XP Pro with VC6 Thanks in advance
Sign In·View Thread·PermaLink

Re: Exe crashes under Vista
Jeff Robertson

3:32 20 Nov '08

In Visual C++ 7.1 it was dieing in ErrorDlg.cpp

I changed the following


void CErrorDlg::OnSize(UINT nType, int cx, int cy)
{
	CDialog::OnSize(nType, cx, cy);
	RECT rc,rc2;
	GetClientRect(&rc);
	rc.bottom -= 30;
	if(m_errorGraph.m_hWndOwner)   // Change HERE 
        {
             m_errorGraph.MoveWindow(&rc);
	    rc2.top = rc.bottom; rc2.left = rc.left; rc2.right   =           rc2.left + 40; rc2.bottom = rc2.top + 30;
	    CWnd* pwnd = GetDlgItem(IDOK);
	    if(pwnd)
            pwnd->MoveWindow(&rc2);
	    m_errorGraph.UpdateGraph();
	    Invalidate();
        }
	// TODO: Add your message handler code here
}

i need a vc6 version
philophos

6:24 29 Aug '07

that is my wish
Sign In·View Thread·PermaLink	3.00/5

mlp
m_kh

10:04 26 Jul '06

hi i want to work on a new mlp subject for my graduation project. of cource i already worked on OCR . plz guide me.
Sign In·View Thread·PermaLink

seems the code forget to free the memory resourses
ninnyknox

12:46 16 Apr '06

this is a very useful class, thanks! but i have two problems: [1] seems the code forget to free the memory resourses [2] why the network converg even slower if i increase the learningRate? thanks you!
Sign In·View Thread·PermaLink	5.00/5

Re: seems the code forget to free the memory resourses
Hossein Khosravi

19:57 16 Apr '06

1. may be. 2. Thats because of Momentum, set m to zero then you can see that by increasing Learning rate at the begining, MLP converges faster.
Sign In·View Thread·PermaLink	2.00/5

Re: need help
Hossein Khosravi

20:25 8 Apr '06

Thats very simple. in fact its the sme as what I done in my demo application, unless you must get inputs from text file rather than screen. assume red points are class -1 and blue points are class +1. other things are the same. good luck.
Sign In·View Thread·PermaLink	2.00/5

Very Large Handwritten Farsi Digit Dataset [modified]
Hossein Khosravi

21:36 15 Jan '06

Hello all

to use this neural network with some real data, there is a free dataset of handwritten farsi digits at this location:
http://www.farsiocr.ir/farsi-digit-dataset[^]
also you can directly contact me(HosseinKhosravi@gmail.com) I will guide you how to access the whole dataset of Farsi digits. This dataset is free for non commercial use.It has more than 102,000 handwritten Persian digits.
GoodLuck;P

Hossein Khosravi

modified on Saturday, August 29, 2009 1:33 AM

Do these NN classes need any change for OCR app?
caykahve

14:33 27 Dec '05

Hello, I'm trying to use your code for an OCR application. I'm facing two problems. 1. I cannot converge to less than 0.15 error 2. After training, I always get the same output value set whatever input set I apply. Have you changed or added something to the NN classes you have used here when you made the Farsi OCR application? Thanks
Sign In·View Thread·PermaLink	2.00/5

Different training set
patrick tran

4:55 13 Sep '05

This is absolutely kool, Andrew. I would like to ask you 2 questions. 1. I would like to reuse the code of NN to train a different dataset which has both nominal and numerical attributes. Is it possible your NN can accept nominal inputs? 2. Is that true that I need only to replace your training data generation by another dataset? And the NN would work just fine? Thanks Patrick
Sign In·View Thread·PermaLink	2.00/5

what about voice recognition
payam govahi

6:46 9 Aug '05

i want to use nural network for recognize 3 sound command but i cannot create nural network like this ,can i use this article to write a program for my purpose ? it is very important for my program about knitting persian rugs ,for now i recogniz sound frequencies by fourier transform (FFT) but i have forced to recognize only frequencies.. regards payam govahi www.payamsoft.com
Sign In·View Thread·PermaLink

Re: what about voice recognition
payamtooba

0:06 21 Jan '09

someone help me
Sign In·View Thread·PermaLink

Doing something very similar
zsnow

11:25 4 Feb '05

As my small research work during my study i am analysing some emulsion diagrams and the thing here i quite similar, except i have 5 colors: picture of training data and predicited values

first i was trying to train my net with standard backpropagation but had a few problems, so i decided to use genetic algorithm for training my net and i got more satisfying results.

A great source of information for me was faq of comp.ai.neural-nets.

questions, comments?

Re: Doing something very similar
sgenie68

2:54 21 Jun '05

Can I get a description of how you did training for your implementation? You can write me off-line to avoid cluttering this thread.
Sign In·View Thread·PermaLink	2.00/5

Thought-provoking
Jerry Evans

12:30 31 Jan '05

Thanks for the article, most interesting. One suggestion: You mention using your backprop system to do OCR for Farsi - it would be great if you could elaborate/describe this process in more detail. This is exactly the kind of discussionthat gets missed from most articles/books. For example how do you format your data ready for input to the MLP and how do you check the recogniser is working correctly ?

Re: Thought-provoking
Hossein Khosravi

21:05 31 Jan '05

at first I extract letters from some handwritten forms, then I did normalization, at last the normalized image(64*64 pixel) was segmented into 8*8 block and gray level average in each block was the value send to each input neuron(64 neuron was at input)... of course there are another features that can be used but the scope of this article is not so that I can describe all my work Confused

. but its interesting that by this easy feature I reached 96% correctness on my training data(12800 letters) and 88% correctness on my test data(3200 letters) OMG

Is that really you Hossien Khosravi??
Anonymous

12:35 30 Jan '05

Khosravi, do you remember me??? I am Damon Chitsaz, your old highschool friend! Remember Javad(Reza Goodarzi??) It has been so long! A weird coincident to meet you here! What is your email address?? Mine is damoncz@hotmail.com Contact me!
Sign In·View Thread·PermaLink

Re: Is that really you Hossien Khosravi??
toxcct

7:02 1 Feb '05

hey, do you remember me ? toxcct, the guy you never met... toxcct@blablabla.com (oops, i forgot, that's not a meeting forum...) TOXCCT >>> GEII power [toxcct][VisualCalc]
Sign In·View Thread·PermaLink

Interesting
Rui A. Rebelo

6:57 30 Jan '05

You code is excellent. It is efficient and easy to understand, provided that the reader knows the teory behind back error propagation neural nets. The program is also very nice, with appealing graphics.
However your article doesn't explain well the theory behind the code. For those willing to understand how the neural nets work I suggest:
http://www.generation5.org/articles.asp?Action=List&Topic=Neural+Networks
It has lots of articles covering back propagation and kohonen neural nets.

Rui A. Rebelo

At last some info...
Anonymous

2:25 30 Jan '05

Excellent, I havent checked the code yet, but at last something that seems to be an easy implementation of a N.Network. I've been working with similar classes but it was a long time ago and I couldnt find anything from that time, so this is great... Cheers
Sign In·View Thread·PermaLink	2.83/5