Click here to Skip to main content
13,287,116 members (44,368 online)
Click here to Skip to main content
Add your own
alternative version


278 bookmarked
Posted 24 Jun 2007

Image Recognition with Neural Networks

, 30 Oct 2007
Rate this:
Please Sign up or sign in to vote.
This article contains a brief description of BackPropagation Artificial Neural Network and its implementation for Image Recognition
Screenshot - screen211.png


Artificial Neural Networks are a recent development tool that are modeled from biological neural networks. The powerful side of this new tool is its ability to solve problems that are very hard to be solved by traditional computing methods (e.g. by algorithms). This work briefly explains Artificial Neural Networks and their applications, describing how to implement a simple ANN for image recognition.


I will try to make the idea clear to the reader who is just interested in the topic.

About Artificial Neural Networks (ANNs)

Artificial Neural Networks (ANNs) are a new approach that follow a different way from traditional computing methods to solve problems. Since conventional computers use algorithmic approach, if the specific steps that the computer needs to follow are not known, the computer cannot solve the problem. That means, traditional computing methods can only solve the problems that we have already understood and knew how to solve. However, ANNs are, in some way, much more powerful because they can solve problems that we do not exactly know how to solve. That's why, of late, their usage is spreading over a wide range of area including, virus detection, robot control, intrusion detection systems, pattern (image, fingerprint, noise..) recognition and so on.

ANNs have the ability to adapt, learn, generalize, cluster or organize data. There are many structures of ANNs including, Percepton, Adaline, Madaline, Kohonen, BackPropagation and many others. Probably, BackPropagation ANN is the most commonly used, as it is very simple to implement and effective. In this work, we will deal with BackPropagation ANNs.

BackPropagation ANNs contain one or more layers each of which are linked to the next layer. The first layer is called the "input layer" which meets the initial input (e.g. pixels from a letter) and so does the last one "output layer" which usually holds the input's identifier (e.g. name of the input letter). The layers between input and output layers are called "hidden layer(s)" which only propagate the previous layer's outputs to the next layer and [back] propagates the following layer's error to the previous layer. Actually, these are the main operations of training a BackPropagation ANN which follows a few steps.

A typical BackPropagation ANN is as depicted below. The black nodes (on the extreme left) are the initial inputs. Training such a network involves two phases. In the first phase, the inputs are propagated forward to compute the outputs for each output node. Then, each of these outputs are subtracted from its desired output, causing an error [an error for each output node]. In the second phase, each of these output errors is passed backward and the weights are fixed. These two phases is continued until the sum of [square of output errors] reaches an acceptable value.

Screenshot - fig1_nnet_thinner.png


The network layers in the figure above are implemented as arrays of structs. The nodes of the layers are implemented as follows:

struct PreInput
    public double Value;
    public double[] Weights;            

struct Input
    public double InputSum;                
    public double Output;                
    public double Error;                
    public double[] Weights;        
struct Hidden        
    public double InputSum;                    
    public double Output;                
    public double Error;                
    public double[] Weights;        
struct Output<T> where T : IComparable<T>         
    public double InputSum;                
    public double output;                
    public double Error;                
    public double Target;     
    public T Value;   

The layers in the figure are implemented as follows (for a three layer network):

private PreInput[] PreInputLayer;
private Input[] InputLayer;
private Hidden[] HiddenLayer;
private Output<string>[] OutputLayer;

Training the network can be summarized as follows:

  • Apply input to the network.
  • Calculate the output.
  • Compare the resulting output with the desired output for the given input. This is called the error.
  • Modify the weights for all neurons using the error.
  • Repeat the process until the error reaches an acceptable value (e.g. error < 1%), which means that the NN was trained successfully, or if we reach a maximum count of iterations, which means that the NN training was not successful.

It is represented as shown below:

void TrainNetwork(TrainingSet,MaxError)
          foreach(Pattern in TrainingSet)
               ForwardPropagate(Pattern);//calculate output 
               BackPropagate()//fix errors, update weights

This is implemented as follows:

public bool Train()
    double currentError = 0;
    int currentIteration = 0;
    NeuralEventArgs Args = new NeuralEventArgs() ;

        currentError = 0;
        foreach (KeyValuePair<T, double[]> p in TrainingSet)
            NeuralNet.ForwardPropagate(p.Value, p.Key);
            currentError += NeuralNet.GetError();
        if (IterationChanged != null && currentIteration % 5 == 0)
            Args.CurrentError = currentError;
            Args.CurrentIteration = currentIteration;
            IterationChanged(this, Args);

    } while (currentError > maximumError && currentIteration < 
    maximumIteration && !Args.Stop);

    if (IterationChanged != null)
        Args.CurrentError = currentError;
        Args.CurrentIteration = currentIteration;
        IterationChanged(this, Args);

    if (currentIteration >= maximumIteration || Args.Stop)   
        return false;//Training Not Successful
    return true;

Where ForwardPropagate(..) and BackPropagate() methods are as shown for a three layer network:

private void ForwardPropagate(double[] pattern, T output)
    int i, j;
    double total;
    //Apply input to the network
    for (i = 0; i < PreInputNum; i++)
        PreInputLayer[i].Value = pattern[i];
    //Calculate The First(Input) Layer's Inputs and Outputs
    for (i = 0; i < InputNum; i++)
        total = 0.0;
        for (j = 0; j < PreInputNum; j++)
            total += PreInputLayer[j].Value * PreInputLayer[j].Weights[i];
        InputLayer[i].InputSum = total;
        InputLayer[i].Output = F(total);
    //Calculate The Second(Hidden) Layer's Inputs and Outputs
    for (i = 0; i < HiddenNum; i++)
        total = 0.0;
        for (j = 0; j < InputNum; j++)
            total += InputLayer[j].Output * InputLayer[j].Weights[i];

        HiddenLayer[i].InputSum = total;
        HiddenLayer[i].Output = F(total);
    //Calculate The Third(Output) Layer's Inputs, Outputs, Targets and Errors
    for (i = 0; i < OutputNum; i++)
        total = 0.0;
        for (j = 0; j < HiddenNum; j++)
            total += HiddenLayer[j].Output * HiddenLayer[j].Weights[i];

        OutputLayer[i].InputSum = total;
        OutputLayer[i].output = F(total);
        OutputLayer[i].Target = OutputLayer[i].Value.CompareTo(output) == 0 ? 1.0 : 0.0;
        OutputLayer[i].Error = (OutputLayer[i].Target - OutputLayer[i].output) *
                                       (OutputLayer[i].output) * (1 - OutputLayer[i].output);
private void BackPropagate()
    int i, j;
    double total;
    //Fix Hidden Layer's Error
    for (i = 0; i < HiddenNum; i++)
        total = 0.0;
        for (j = 0; j < OutputNum; j++)
            total += HiddenLayer[i].Weights[j] * OutputLayer[j].Error;
        HiddenLayer[i].Error = total;
    //Fix Input Layer's Error
    for (i = 0; i < InputNum; i++)
        total = 0.0;
        for (j = 0; j < HiddenNum; j++)
            total += InputLayer[i].Weights[j] * HiddenLayer[j].Error;
        InputLayer[i].Error = total;
    //Update The First Layer's Weights
    for (i = 0; i < InputNum; i++)
        for(j = 0; j < PreInputNum; j++)
            PreInputLayer[j].Weights[i] +=
                LearningRate * InputLayer[i].Error * PreInputLayer[j].Value;
    //Update The Second Layer's Weights
    for (i = 0; i < HiddenNum; i++)
        for (j = 0; j < InputNum; j++)
            InputLayer[j].Weights[i] +=
                LearningRate * HiddenLayer[i].Error * InputLayer[j].Output;
    //Update The Third Layer's Weights
    for (i = 0; i < OutputNum; i++)
        for (j = 0; j < HiddenNum; j++)
            HiddenLayer[j].Weights[i] +=
                LearningRate * OutputLayer[i].Error * HiddenLayer[j].Output;

Testing the App

The program trains the network using bitmap images that are located in a folder. This folder must be in the following format:

  • There must be one (input) folder that contains input images [*.bmp].
  • Each image's name is the target (or output) value for the network (the pixel values of the image are the inputs, of course) .

As testing the classes requires to train the network first, there must be a folder in this format. "PATTERNS" and "ICONS" folders [depicted below] in the Debug folder fit this format.

Screenshot - fig2_sampleInput_thinner.png Screenshot - fig3_sampleInput_thinner.png


  • 30th September, 2007: Simplified the app
  • 24th June, 2007: Initial Release

References & External Links


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

Murat Firat
Software Developer (Senior)
Turkey Turkey
Has BS degree on computer science, working as software engineer in istanbul.

You may also be interested in...


Comments and Discussions

Questionwhat is the convergence? Pin
onuriztech7-Apr-09 10:27
memberonuriztech7-Apr-09 10:27 
AnswerRe: what is the convergence? Pin
Murat Firat8-Apr-09 2:20
memberMurat Firat8-Apr-09 2:20 
QuestionCurrent Error Pin
jimbobmcgee30-Mar-09 10:00
memberjimbobmcgee30-Mar-09 10:00 
Generalproblem running and compiling Pin
prophet8617-Mar-09 9:15
memberprophet8617-Mar-09 9:15 
GeneralRe: problem running and compiling Pin
zimpzon15-Mar-10 23:37
memberzimpzon15-Mar-10 23:37 
General[Message Deleted] Pin
Member 453077115-Mar-09 4:15
memberMember 453077115-Mar-09 4:15 
GeneralCode in VB.Net Pin
mu'a7-Mar-09 10:44
membermu'a7-Mar-09 10:44 
Questionwhat can i do to increase acuracy? Pin
Member 45307713-Mar-09 5:00
memberMember 45307713-Mar-09 5:00 
AnswerRe: what can i do to increase acuracy? Pin
SafarTimura9-May-09 13:21
memberSafarTimura9-May-09 13:21 
GeneralUnhandled exception Pin
CristinaF8-Dec-08 8:41
memberCristinaF8-Dec-08 8:41 
GeneralRe: Unhandled exception Pin
CristinaF8-Dec-08 9:19
memberCristinaF8-Dec-08 9:19 
QuestionClassification Image....? Pin
demonlove7-Dec-08 5:04
memberdemonlove7-Dec-08 5:04 
AnswerRe: Classification Image....? Pin
SafarTimura9-May-09 13:32
memberSafarTimura9-May-09 13:32 
Questionis this feed forward neural network? Pin
Member 453077129-Nov-08 21:35
memberMember 453077129-Nov-08 21:35 
QuestionTraining takes so long time, any idea? Pin
tulipvn18-Nov-08 1:26
membertulipvn18-Nov-08 1:26 
AnswerRe: Training takes so long time, any idea? Pin
Murat Firat21-Nov-08 8:54
memberMurat Firat21-Nov-08 8:54 
GeneralRe: Training takes so long time, any idea? Pin
Himdevi27-Mar-12 5:20
memberHimdevi27-Mar-12 5:20 
hey murat..great work
jst help meout i gt stuck in one problem
first of all i want to knw dat the images used by u in ur project comprises of hw many pixels?
i mean wats their size?
n in ur project there are many images so wat algo u hv used?
u hv multiplied d size of image with inputs or anythng else?
GeneralCompiling Error in Visual C# 2008 Express Pin
mangotj17-Oct-08 17:02
membermangotj17-Oct-08 17:02 
Questionrecognition Pin
brimzi13-Oct-08 0:59
memberbrimzi13-Oct-08 0:59 
GeneralQuestion [modified] Pin
tulipvn8-Oct-08 1:41
membertulipvn8-Oct-08 1:41 
QuestionJava version? Pin
tulipvn23-Sep-08 5:43
membertulipvn23-Sep-08 5:43 
AnswerRe: Java version? Pin
Murat Firat27-Sep-08 11:37
memberMurat Firat27-Sep-08 11:37 
GeneralImage size and accuracy Pin
visusnet22-Sep-08 12:00
membervisusnet22-Sep-08 12:00 
GeneralRe: Image size and accuracy Pin
Murat Firat27-Sep-08 11:32
memberMurat Firat27-Sep-08 11:32 
GeneralCalculating output layer error Pin
jack_wind11-Sep-08 23:32
memberjack_wind11-Sep-08 23:32 
GeneralRe: Calculating output layer error Pin
Murat Firat18-Sep-08 6:09
memberMurat Firat18-Sep-08 6:09 
Questionhowto Apply BackPropagation in FingerPrint image matching? Pin
swdev.bali27-Aug-08 1:22
memberswdev.bali27-Aug-08 1:22 
AnswerRe: howto Apply BackPropagation in FingerPrint image matching? Pin
Murat Firat1-Sep-08 4:52
memberMurat Firat1-Sep-08 4:52 
GeneralRe: howto Apply BackPropagation in FingerPrint image matching? Pin
swdev.bali1-Sep-08 17:44
memberswdev.bali1-Sep-08 17:44 
GeneralRe: howto Apply BackPropagation in FingerPrint image matching? Pin
Murat Firat2-Sep-08 17:36
memberMurat Firat2-Sep-08 17:36 
QuestionGreat aplication - Generalization Pin
ignacio.7811-Aug-08 13:58
memberignacio.7811-Aug-08 13:58 
AnswerRe: Great aplication - Generalization Pin
Murat Firat15-Aug-08 18:54
memberMurat Firat15-Aug-08 18:54 
GeneralUnable to download Pin
shery_sa9-Jul-08 9:53
membershery_sa9-Jul-08 9:53 
QuestionImageProcessing.ToMatrix Pin
denny_cucu14-Jun-08 23:30
memberdenny_cucu14-Jun-08 23:30 
AnswerRe: ImageProcessing.ToMatrix Pin
Murat Firat17-Jun-08 11:05
memberMurat Firat17-Jun-08 11:05 
GeneralRe: ImageProcessing.ToMatrix Pin
denny_cucu28-Jun-08 23:25
memberdenny_cucu28-Jun-08 23:25 
GeneralRe: ImageProcessing.ToMatrix Pin
Himdevi24-Mar-12 21:18
memberHimdevi24-Mar-12 21:18 
GeneralRe: ImageProcessing.ToMatrix Pin
Murat Firat25-Mar-12 12:55
memberMurat Firat25-Mar-12 12:55 
GeneralRe: ImageProcessing.ToMatrix Pin
Himdevi26-Mar-12 5:22
memberHimdevi26-Mar-12 5:22 
QuestionImage Detection Pin
YehudaG12-May-08 5:12
memberYehudaG12-May-08 5:12 
Questionwhat is the initial value of output layer??? Pin
sandipmuk7-May-08 22:18
membersandipmuk7-May-08 22:18 
AnswerRe: what is the initial value of output layer??? Pin
Murat Firat11-May-08 21:22
memberMurat Firat11-May-08 21:22 
Questionwhats the network archeticture? Pin
jamilkhan00723-Apr-08 22:46
memberjamilkhan00723-Apr-08 22:46 
GeneralRe: whats the network archeticture? Pin
Murat Firat23-Apr-08 23:32
memberMurat Firat23-Apr-08 23:32 
QuestionRe: whats the network archeticture? Pin
jamilkhan00727-Apr-08 0:52
memberjamilkhan00727-Apr-08 0:52 
GeneralRe: whats the network archeticture? Pin
Murat Firat27-Apr-08 12:05
memberMurat Firat27-Apr-08 12:05 
GeneralRe: whats the network archeticture? Pin
jamilkhan0072-May-08 23:29
memberjamilkhan0072-May-08 23:29 
QuestionFace Detection Nueral Network? Pin
jamilkhan00718-Apr-08 2:53
memberjamilkhan00718-Apr-08 2:53 
QuestionImage Recognition Advanced Engine Development Pin
ankswe15-Apr-08 4:51
memberankswe15-Apr-08 4:51 
GeneralExcellent! Pin
newbie0827-Mar-08 2:00
membernewbie0827-Mar-08 2:00 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web04 | 2.8.171207.1 | Last Updated 30 Oct 2007
Article Copyright 2007 by Murat Firat
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid