11,412,516 members (73,999 online)

# Image Recognition with Neural Networks

, 30 Oct 2007 CPOL
 Rate this:
This article contains a brief description of BackPropagation Artificial Neural Network and its implementation for Image Recognition

## Introduction

Artificial Neural Networks are a recent development tool that are modeled from biological neural networks. The powerful side of this new tool is its ability to solve problems that are very hard to be solved by traditional computing methods (e.g. by algorithms). This work briefly explains Artificial Neural Networks and their applications, describing how to implement a simple ANN for image recognition.

## Background

I will try to make the idea clear to the reader who is just interested in the topic.

## About Artificial Neural Networks (ANNs)

Artificial Neural Networks (ANNs) are a new approach that follow a different way from traditional computing methods to solve problems. Since conventional computers use algorithmic approach, if the specific steps that the computer needs to follow are not known, the computer cannot solve the problem. That means, traditional computing methods can only solve the problems that we have already understood and knew how to solve. However, ANNs are, in some way, much more powerful because they can solve problems that we do not exactly know how to solve. That's why, of late, their usage is spreading over a wide range of area including, virus detection, robot control, intrusion detection systems, pattern (image, fingerprint, noise..) recognition and so on.

ANNs have the ability to adapt, learn, generalize, cluster or organize data. There are many structures of ANNs including, Percepton, Adaline, Madaline, Kohonen, BackPropagation and many others. Probably, BackPropagation ANN is the most commonly used, as it is very simple to implement and effective. In this work, we will deal with BackPropagation ANNs.

BackPropagation ANNs contain one or more layers each of which are linked to the next layer. The first layer is called the "input layer" which meets the initial input (e.g. pixels from a letter) and so does the last one "output layer" which usually holds the input's identifier (e.g. name of the input letter). The layers between input and output layers are called "hidden layer(s)" which only propagate the previous layer's outputs to the next layer and [back] propagates the following layer's error to the previous layer. Actually, these are the main operations of training a BackPropagation ANN which follows a few steps.

A typical BackPropagation ANN is as depicted below. The black nodes (on the extreme left) are the initial inputs. Training such a network involves two phases. In the first phase, the inputs are propagated forward to compute the outputs for each output node. Then, each of these outputs are subtracted from its desired output, causing an error [an error for each output node]. In the second phase, each of these output errors is passed backward and the weights are fixed. These two phases is continued until the sum of [square of output errors] reaches an acceptable value.

## Implementation

The network layers in the figure above are implemented as arrays of structs. The nodes of the layers are implemented as follows:

```[Serializable]
struct PreInput
{
public double Value;
public double[] Weights;
};

[Serializable]
struct Input
{
public double InputSum;
public double Output;
public double Error;
public double[] Weights;
};

[Serializable]
struct Hidden
{
public double InputSum;
public double Output;
public double Error;
public double[] Weights;
};

[Serializable]
struct Output<T> where T : IComparable<T>
{
public double InputSum;
public double output;
public double Error;
public double Target;
public T Value;
};```

The layers in the figure are implemented as follows (for a three layer network):

```private PreInput[] PreInputLayer;
private Input[] InputLayer;
private Hidden[] HiddenLayer;
private Output<string>[] OutputLayer;```

Training the network can be summarized as follows:

• Apply input to the network.
• Calculate the output.
• Compare the resulting output with the desired output for the given input. This is called the error.
• Modify the weights for all neurons using the error.
• Repeat the process until the error reaches an acceptable value (e.g. error < 1%), which means that the NN was trained successfully, or if we reach a maximum count of iterations, which means that the NN training was not successful.

It is represented as shown below:

```void TrainNetwork(TrainingSet,MaxError)
{
while(CurrentError>MaxError)
{
foreach(Pattern in TrainingSet)
{
ForwardPropagate(Pattern);//calculate output
BackPropagate()//fix errors, update weights
}
}
}```

This is implemented as follows:

```public bool Train()
{
double currentError = 0;
int currentIteration = 0;
NeuralEventArgs Args = new NeuralEventArgs() ;

do
{
currentError = 0;
foreach (KeyValuePair<T, double[]> p in TrainingSet)
{
NeuralNet.ForwardPropagate(p.Value, p.Key);
NeuralNet.BackPropagate();
currentError += NeuralNet.GetError();
}

currentIteration++;

if (IterationChanged != null && currentIteration % 5 == 0)
{
Args.CurrentError = currentError;
Args.CurrentIteration = currentIteration;
IterationChanged(this, Args);
}

} while (currentError > maximumError && currentIteration <
maximumIteration && !Args.Stop);

if (IterationChanged != null)
{
Args.CurrentError = currentError;
Args.CurrentIteration = currentIteration;
IterationChanged(this, Args);
}

if (currentIteration >= maximumIteration || Args.Stop)
return false;//Training Not Successful

return true;
}```

Where `ForwardPropagate(..)` and `BackPropagate()` methods are as shown for a three layer network:

```private void ForwardPropagate(double[] pattern, T output)
{
int i, j;
double total;
//Apply input to the network
for (i = 0; i < PreInputNum; i++)
{
PreInputLayer[i].Value = pattern[i];
}
//Calculate The First(Input) Layer's Inputs and Outputs
for (i = 0; i < InputNum; i++)
{
total = 0.0;
for (j = 0; j < PreInputNum; j++)
{
total += PreInputLayer[j].Value * PreInputLayer[j].Weights[i];
}
InputLayer[i].InputSum = total;
InputLayer[i].Output = F(total);
}
//Calculate The Second(Hidden) Layer's Inputs and Outputs
for (i = 0; i < HiddenNum; i++)
{
total = 0.0;
for (j = 0; j < InputNum; j++)
{
total += InputLayer[j].Output * InputLayer[j].Weights[i];
}

HiddenLayer[i].InputSum = total;
HiddenLayer[i].Output = F(total);
}
//Calculate The Third(Output) Layer's Inputs, Outputs, Targets and Errors
for (i = 0; i < OutputNum; i++)
{
total = 0.0;
for (j = 0; j < HiddenNum; j++)
{
total += HiddenLayer[j].Output * HiddenLayer[j].Weights[i];
}

OutputLayer[i].InputSum = total;
OutputLayer[i].output = F(total);
OutputLayer[i].Target = OutputLayer[i].Value.CompareTo(output) == 0 ? 1.0 : 0.0;
OutputLayer[i].Error = (OutputLayer[i].Target - OutputLayer[i].output) *
(OutputLayer[i].output) * (1 - OutputLayer[i].output);
}
}

private void BackPropagate()
{
int i, j;
double total;
//Fix Hidden Layer's Error
for (i = 0; i < HiddenNum; i++)
{
total = 0.0;
for (j = 0; j < OutputNum; j++)
{
total += HiddenLayer[i].Weights[j] * OutputLayer[j].Error;
}
HiddenLayer[i].Error = total;
}
//Fix Input Layer's Error
for (i = 0; i < InputNum; i++)
{
total = 0.0;
for (j = 0; j < HiddenNum; j++)
{
total += InputLayer[i].Weights[j] * HiddenLayer[j].Error;
}
InputLayer[i].Error = total;
}
//Update The First Layer's Weights
for (i = 0; i < InputNum; i++)
{
for(j = 0; j < PreInputNum; j++)
{
PreInputLayer[j].Weights[i] +=
LearningRate * InputLayer[i].Error * PreInputLayer[j].Value;
}
}
//Update The Second Layer's Weights
for (i = 0; i < HiddenNum; i++)
{
for (j = 0; j < InputNum; j++)
{
InputLayer[j].Weights[i] +=
LearningRate * HiddenLayer[i].Error * InputLayer[j].Output;
}
}
//Update The Third Layer's Weights
for (i = 0; i < OutputNum; i++)
{
for (j = 0; j < HiddenNum; j++)
{
HiddenLayer[j].Weights[i] +=
LearningRate * OutputLayer[i].Error * HiddenLayer[j].Output;
}
}
}```

## Testing the App

The program trains the network using bitmap images that are located in a folder. This folder must be in the following format:

• There must be one (input) folder that contains input images [*.bmp].
• Each image's name is the target (or output) value for the network (the pixel values of the image are the inputs, of course) .

As testing the classes requires to train the network first, there must be a folder in this format. "PATTERNS" and "ICONS" folders [depicted below] in the Debug folder fit this format.

## History

• 30th September, 2007: Simplified the app
• 24th June, 2007: Initial Release

## Share

Software Developer (Senior)
Turkey
Has BS degree on CS, working as SW engineer at istanbul.

 First PrevNext
 Ask [file trainning] Member 10648205 at 14-Jan-15 21:59 Member 10648205 14-Jan-15 21:59
 My vote of 3 asgharmalik at 29-Oct-14 1:29 asgharmalik 29-Oct-14 1:29
 I can't run the programm Member 11023549 at 19-Aug-14 22:22 Member 11023549 19-Aug-14 22:22
 many thanks for this topic rami_ka at 7-Apr-14 13:57 rami_ka 7-Apr-14 13:57
 emotional expressions Member 9538830 at 5-Feb-14 10:11 Member 9538830 5-Feb-14 10:11
 Output and Input Neuron Number Arindom_chanda at 26-Aug-13 23:29 Arindom_chanda 26-Aug-13 23:29
 My vote of 1 MAN2MAN at 25-Dec-12 0:49 MAN2MAN 25-Dec-12 0:49
 how big my images? amin_nmer at 1-Dec-12 23:58 amin_nmer 1-Dec-12 23:58
 My vote of 4 omyildirim at 30-Nov-12 3:39 omyildirim 30-Nov-12 3:39
 [IMPORTANT] The implementation contains MAJOR BUGS giuseppemag at 28-Jun-12 0:32 giuseppemag 28-Jun-12 0:32
 Re: [IMPORTANT] The implementation contains MAJOR BUGS sachitha12345 at 5-Nov-12 21:26 sachitha12345 5-Nov-12 21:26
 matching algorithm Member 8138227 at 21-Apr-12 20:19 Member 8138227 21-Apr-12 20:19
 My vote of 5 praneshkmr at 24-Jan-12 19:45 praneshkmr 24-Jan-12 19:45
 Entering and outputting more than one character ibnkhaldun at 2-Jan-12 11:41 ibnkhaldun 2-Jan-12 11:41
 input value not the same as output value Member 3913283 at 3-Dec-11 6:22 Member 3913283 3-Dec-11 6:22
 My vote of 3 HansiHermann at 28-Oct-11 4:12 HansiHermann 28-Oct-11 4:12
 flow chart baguswahyu at 27-Oct-11 3:38 baguswahyu 27-Oct-11 3:38
 Logic behind the programm Member 8203782 at 13-Sep-11 21:35 Member 8203782 13-Sep-11 21:35
 little confused about PreInput and Input swarajs at 7-Aug-11 19:05 swarajs 7-Aug-11 19:05
 Great Work:-) chikkisherry at 20-Jul-11 21:56 chikkisherry 20-Jul-11 21:56
 Increase probability RonZohan at 20-Jul-11 4:30 RonZohan 20-Jul-11 4:30
 How can i increase the probability of getting the desired output of the image? i have been training lately but the outcome wouldn't go as high as 50%
 Problem with self coding spider853 at 9-Jul-11 16:10 spider853 9-Jul-11 16:10
 Need your guidance... apepe at 23-Jun-11 17:55 apepe 23-Jun-11 17:55
 Re: Need your guidance... Murat Firat at 24-Jun-11 9:56 Murat Firat 24-Jun-11 9:56
 Re: Need your guidance... apepe at 25-Jun-11 17:32 apepe 25-Jun-11 17:32
 Re: Need your guidance... Murat Firat at 27-Jun-11 1:51 Murat Firat 27-Jun-11 1:51
 help please!!!!! asmita5 at 29-May-11 8:10 asmita5 29-May-11 8:10
 Re: help please!!!!! Bikash_coder at 16-Jun-11 0:09 Bikash_coder 16-Jun-11 0:09
 Re: help please!!!!! asmita5 at 16-Jun-11 2:48 asmita5 16-Jun-11 2:48
 Re: help please!!!!! Murat Firat at 16-Jun-11 3:17 Murat Firat 16-Jun-11 3:17
 Question about Number of Input Unit & Number of Hiden Unit wiswadipa at 27-May-11 3:44 wiswadipa 27-May-11 3:44
 How do I change to recognize the binary pattern shamlen at 12-May-11 17:30 shamlen 12-May-11 17:30
 Re: How do I change to recognize the binary pattern merovingian18 at 29-Mar-12 5:47 merovingian18 29-Mar-12 5:47
 How i can increase hidden node in BP1Layer.cs Bikash_coder at 16-Feb-11 23:45 Bikash_coder 16-Feb-11 23:45
 Re: How i can increase hidden node in BP1Layer.cs Murat Firat at 20-Feb-11 21:35 Murat Firat 20-Feb-11 21:35
 layer 3 better or not Member 1964241 at 10-Feb-11 22:17 Member 1964241 10-Feb-11 22:17
 Re: layer 3 better or not Murat Firat at 20-Feb-11 21:34 Murat Firat 20-Feb-11 21:34
 great work... :) kireina_3012 at 29-Jan-11 11:00 kireina_3012 29-Jan-11 11:00
 Re: great work... :) Bikash_coder at 30-Jan-11 1:17 Bikash_coder 30-Jan-11 1:17
 Re: great work... :) kireina_3012 at 1-Feb-11 8:59 kireina_3012 1-Feb-11 8:59
 Re: great work... :) Murat Firat at 2-Feb-11 10:42 Murat Firat 2-Feb-11 10:42
 Target output SeasickSailor at 20-Dec-10 3:14 SeasickSailor 20-Dec-10 3:14
 Displaying the image [modified] swathi6589 at 8-Oct-10 3:56 swathi6589 8-Oct-10 3:56
 Finger print Reconization waqas munim at 27-Jul-10 8:02 waqas munim 27-Jul-10 8:02
 blood cell images recognition [modified] kushagra.thakur at 11-Jul-10 17:36 kushagra.thakur 11-Jul-10 17:36
 Re: blood cell images recognition Murat Firat at 12-Jul-10 9:00 Murat Firat 12-Jul-10 9:00
 question diedou at 21-Jun-10 4:41 diedou 21-Jun-10 4:41
 Last Visit: 31-Dec-99 19:00     Last Update: 26-Apr-15 16:17 Refresh 1234 Next »