|
|||||||||||||||||||||||
|
|||||||||||||||||||||||
|
Announcements
Services
Chapters
Feature Zones
|
Note: This is an unedited contribution. If this article is inappropriate,
needs attention or copies someone else's work without reference then please
Report This Article
IntroductionArtificial Neural Networks are a recent development tool
that are modeled from biological neural networks. The powerful side of this new
tool is its ability to solve problems that are very hard to be solved by
traditional computing methods(e.g. by algorithms). This work briefly explains
Artificial Neural Networks and their applications, describing how to implement
a simple ANN for image recogniton. BackgroundI will try to make the idea clear to the reader who is just interested in the topic.
About Artificial Neural Networks (ANNs)
Artificial Neural Networks (ANNs) are a new approach that
follows a different way from traditional computing methods to solve problems.
Since conventional computers use algorithmic approach, if the specific steps
that the computer needs to follow are not known, the computer can not solve the
problem. That means, traditional computing methods can only solve the problems
that we have already understood and knew how to solve. However, ANNs are, in
some way, much more powerful because they can solve problems that we do not
exactly know how to solve. That's why their usage are recently spreading over a
wide range of area including, virus detection, robot control, intrusion
detection systems, pattern (image, fingerprint, noise..) recognition and so on. ANNs has the ability to adapt, learn, generalise, cluster or
organise data. There are so many structures of ANNs including, Percepton,
Adaline, Madaline, Kohonen, BackPropagation and many others. Probably, BackPropagation
ANNs is the most commonly used, as it is very simple to implement and effective.
In this work we will deal with BackPropagation ANNs. A typical BackPropagation ANN is as depicted below. The black nodes (on the leftest) are the initial inputs. Training such a network involves two phases. In the first phase, the inputs are propagated forward to compute the outputs for each output node. Then, each of these outputs are substracted from its desired output, causing an error [an error for each output node]. In the second phase, each of these output errors is passed backward and the weights are fixed. These two phases is continued until sum of [square of output errors] reaches to an acceptable value.
![]()
ImplementationThe network layers on the figure above are implemented as arrays of structs. The nodes of the layers are implemented as:
[Serializable]
struct PreInput
{
public double Value;
public double[] Weights;
};
[Serializable]
struct Input
{
public double InputSum;
public double Output;
public double Error;
public double[] Weights;
};
[Serializable]
struct Hidden
{
public double InputSum;
public double Output;
public double Error;
public double[] Weights;
};
[Serializable]
struct Output<T> where T : IComparable<T>
{
public double InputSum;
public double output;
public double Error;
public double Target;
public T Value;
};
And the layers on the figure are implemented as (for three layer network): private PreInput[] PreInputLayer;
private Input[] InputLayer;
private Hidden[] HiddenLayer;
private Output<string>[] OutputLayer;
Training the network can be summarized as followed:
It is represented as: void TrainNetwork(TrainingSet,MaxError)
{
while(CurrentError>MaxError)
{
foreach(Pattern in TrainingSet)
{
ForwardPropagate(Pattern);//calculate output
BackPropagate()//fix errors, update weights
}
}
}
public bool Train()
{
double currentError = 0;
int currentIteration = 0;
NeuralEventArgs Args = new NeuralEventArgs() ;
do
{
currentError = 0;
foreach (KeyValuePair<T, double[]> p in TrainingSet)
{
NeuralNet.ForwardPropagate(p.Value, p.Key);
NeuralNet.BackPropagate();
currentError += NeuralNet.GetError();
}
currentIteration++;
if (IterationChanged != null && currentIteration % 5 == 0)
{
Args.CurrentError = currentError;
Args.CurrentIteration = currentIteration;
IterationChanged(this, Args);
}
} while (currentError > maximumError && currentIteration < maximumIteration && !Args.Stop);
if (IterationChanged != null)
{
Args.CurrentError = currentError;
Args.CurrentIteration = currentIteration;
IterationChanged(this, Args);
}
if (currentIteration >= maximumIteration || Args.Stop)
return false;//Training Not Successful
return true;
}
private void ForwardPropagate(double[] pattern, T output)
{
int i, j;
double total;
//Apply input to the network
for (i = 0; i < PreInputNum; i++)
{
PreInputLayer[i].Value = pattern[i];
}
//Calculate The First(Input) Layer's Inputs and Outputs
for (i = 0; i < InputNum; i++)
{
total = 0.0;
for (j = 0; j < PreInputNum; j++)
{
total += PreInputLayer[j].Value * PreInputLayer[j].Weights[i];
}
InputLayer[i].InputSum = total;
InputLayer[i].Output = F(total);
}
//Calculate The Second(Hidden) Layer's Inputs and Outputs
for (i = 0; i < HiddenNum; i++)
{
total = 0.0;
for (j = 0; j < InputNum; j++)
{
total += InputLayer[j].Output * InputLayer[j].Weights[i];
}
HiddenLayer[i].InputSum = total;
HiddenLayer[i].Output = F(total);
}
//Calculate The Third(Output) Layer's Inputs, Outputs, Targets and Errors
for (i = 0; i < OutputNum; i++)
{
total = 0.0;
for (j = 0; j < HiddenNum; j++)
{
total += HiddenLayer[j].Output * HiddenLayer[j].Weights[i];
}
OutputLayer[i].InputSum = total;
OutputLayer[i].output = F(total);
OutputLayer[i].Target = OutputLayer[i].Value.CompareTo(output) == 0 ? 1.0 : 0.0;
OutputLayer[i].Error = (OutputLayer[i].Target - OutputLayer[i].output) *
(OutputLayer[i].output) * (1 - OutputLayer[i].output);
}
}
private void BackPropagate()
{
int i, j;
double total;
//Fix Hidden Layer's Error
for (i = 0; i < HiddenNum; i++)
{
total = 0.0;
for (j = 0; j < OutputNum; j++)
{
total += HiddenLayer[i].Weights[j] * OutputLayer[j].Error;
}
HiddenLayer[i].Error = total;
}
//Fix Input Layer's Error
for (i = 0; i < InputNum; i++)
{
total = 0.0;
for (j = 0; j < HiddenNum; j++)
{
total += InputLayer[i].Weights[j] * HiddenLayer[j].Error;
}
InputLayer[i].Error = total;
}
//Update The First Layer's Weights
for (i = 0; i < InputNum; i++)
{
for(j = 0; j < PreInputNum; j++)
{
PreInputLayer[j].Weights[i] +=
LearningRate * InputLayer[i].Error * PreInputLayer[j].Value;
}
}
//Update The Second Layer's Weights
for (i = 0; i < HiddenNum; i++)
{
for (j = 0; j < InputNum; j++)
{
InputLayer[j].Weights[i] +=
LearningRate * HiddenLayer[i].Error * InputLayer[j].Output;
}
}
//Update The Third Layer's Weights
for (i = 0; i < OutputNum; i++)
{
for (j = 0; j < HiddenNum; j++)
{
HiddenLayer[j].Weights[i] +=
LearningRate * OutputLayer[i].Error * HiddenLayer[j].Output;
}
}
}
The program trains the network using bitmap images that are located in a folder. This folder must be in the following format: |
||||||||||||||||||||||
|
PermaLink |
Privacy |
Terms
of Use
Last Updated: 30 Oct 2007 Editor: |
Copyright 2007 by Murat Firat Everything else Copyright © CodeProject, 1999-2008 Web09 | Advertise on the Code Project |