Click here to Skip to main content
11,932,681 members (61,166 online)
Click here to Skip to main content
Add your own
alternative version

Tagged as


58 bookmarked

Single Layer Perceptron as Linear Classifier

, 7 Nov 2010 CPOL
Rate this:
Please Sign up or sign in to vote.
In this article, I will show you how to use single layer percetron as linear classifier of 2 classes.


Perceptron is the simplest type of feed forward neural network. It was designed by Frank Rosenblatt as dichotomic classifier of two classes which are linearly separable. This means that the type of problems the network can solve must be linearly separable. Basic perceptron consists of 3 layers:

  • Sensor layer
  • Associative layer
  • Output neuron

There are a number of inputs (xn) in sensor layer, weights (wn) and an output. Sometimes w0 is called bias and x0 = +1/-1 (In this case is x0=-1).


For every input on the perceptron (including bias), there is a corresponding weight. To calculate the output of the perceptron, every input is multiplied by its corresponding weight. Then weighted sum is computed of all inputs and fed through a limiter function that evaluates the final output of the perceptron.

The output of neuron is formed by activation of the output neuron, which is function of input:

(1) eq1.JPG

The activation function F can be linear so that we have a linear network, or nonlinear. In this example, I decided to use threshold (signum) function:

(2) eq2.JPG

Output of network in this case is either +1 or -1 depending on the input. If the total input (weighted sum of all inputs) is positive, then the pattern belongs to class +1, otherwise to class -1. Because of this behavior, we can use perceptron for classification tasks.

Let's consider we have a perceptron with 2 inputs and we want to separate input patterns into 2 classes. In this case, the separation between the classes is straight line, given by equation:

(3) eq3.gif

When we set x0=-1 and mark w0=?, then we can rewrite equation (3) into form:

(4) eq4.gif

Here I will describe the learning method for perceptron. Learning method of perceptron is an iterative procedure that adjust the weights. A learning sample is presented to the network. For each weight, the new value is computed by adding a correction to the old value. The threshold is updated in the same way:

(5) eq5.gif


where y is output of perceptron, d is desired output and ? is the learning parameter.

Using the Program

When you run the program, you see area where you can input samples. Clicking by left button on this area, you will add first class sample (blue cross). Clicking by right button on this area, you will add first class sample (red cross). Samples are added to the samples list. You can also set learning rate and number of iterations. When you have set all these values, you can click on Learn button to start learning.

Using the Code

All samples are stored in generic list samples which holds only Sample class objects.

public class Sample
    double x1;
    double x2;
    double cls;

    public Sample(double x1, double x2, int cls)
        this.x1 = x1;
        this.x2 = x2;
        this.cls = cls;

    public double X1
        get { return x1; }
        set { this.x1 = value; }

    public double X2
        get { return x2; }
        set { this.x2 = value; }

    public double Class
        get { return cls; }
        set { this.cls = value; }

Before running a learning of perceptron is important to set learning rate and number of iterations. Perceptron has one great property. If solution exists, perceptron always find it but problem occurs, when solution does not exist. In this case, perceptron will try to find the solution in infinity loop and to avoid this, it is better to set maximum number of iterations.

The next step is to assign random values for weights (w0, w1 and w2).

Random rnd = new Random();

w0 = rnd.NextDouble();
w1 = rnd.NextDouble();
w2 = rnd.NextDouble();

When random values are assigned to weights, we can loop through samples and compute output for every sample and compare it with desired output.

double x1 = samples[i].X1;
double x2 = samples[i].X2;
int y;

if (((w1 * x1) + (w2 * x2) - w0) < 0)
    y = -1;
    y = 1;

I decided to set x0=-1 and for this reason, the output of perceptron is given by equation: y=w1*w1+w2*w2-w0. When perceptron output and desired output doesn’t match, we must compute new weights:

if (y != samples[i].Class)
    error = true;

    w0 = w0 + alpha * (samples[i].Class - y) * x0 / 2;
    w1 = w1 + alpha * (samples[i].Class - y) * x1 / 2;
    w2 = w2 + alpha * (samples[i].Class - y) * x2 / 2;

Y is output of perceptron and samples[i].Class is desired output. The last 2 steps (looping through samples and computing new weights), we must repeat while the error variable is <> 0 and current number of iterations (iterations) is less than maxIterations.

int i;
int iterations = 0;
bool error = true;

maxIterations = int.Parse(txtIterations.Text);

Random rnd = new Random();

w0 = rnd.NextDouble();
w1 = rnd.NextDouble();
w2 = rnd.NextDouble();

alpha = (double)trackLearningRate.Value / 1000;

while (error && iterations < maxIterations)
    error = false;

    for (i = 0; i <= samples.Count - 1; i++)
        double x1 = samples[i].X1;
        double x2 = samples[i].X2;
        int y;

        if (((w1 * x1) + (w2 * x2) - w0) < 0)
            y = -1;
            y = 1;

        if (y != samples[i].Class)
            error = true;

            w0 = w0 + alpha * (samples[i].Class - y) * x0 / 2; 
            w1 = w1 + alpha * (samples[i].Class - y) * x1 / 2;
            w2 = w2 + alpha * (samples[i].Class - y) * x2 / 2;

Function DrawSeparationLine draws separation line of 2 classes.


  • 07 Nov 2010 - Original version posted


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

Kanasz Robert
Architect The Staffing Edge & Marwin Cassovia Soft
Slovakia Slovakia
My name is Robert Kanasz and I have been working with ASP.NET, WinForms and C# for several years.
MCSD - Web Applications
MCSE - Data Platform
MCPD - ASP.NET Developer 3.5
- Web Developer 4
MCITP - Database Administrator 2008
- Database Developer 2008
MCSA - SQL Server 2012
MCTS - .NET Framework 3.5, ASP.NET Applications
- SQL Server 2008, Database Development
- SQL Server 2008, Implementation and Maintenance
- .NET Framework 4, Data Access
- .NET Framework 4, Service Communication Applications
- .NET Framework 4, Web Applications
MS - Programming in HTML5 with JavaScript and CSS3 Specialist

Open source projects: DBScripter - Library for scripting SQL Server database objects

Please, do not forget vote

You may also be interested in...

Comments and Discussions

QuestionHow to Draw Separation Line Pin
Member 1126450520-Apr-15 16:33
memberMember 1126450520-Apr-15 16:33 
Generalgood Pin
Omar Gameel Salem30-Sep-14 19:50
professionalOmar Gameel Salem30-Sep-14 19:50 
Questionperceptron Pin
Member 109267945-Jul-14 15:47
memberMember 109267945-Jul-14 15:47 
QuestionIncorrectly dividing by 2 Pin
twocs25-Nov-12 17:39
membertwocs25-Nov-12 17:39 
QuestionNice one!!! Pin
strucker_luc18-Nov-12 4:17
memberstrucker_luc18-Nov-12 4:17 
AnswerRe: Nice one!!! Pin
Kanasz Robert18-Nov-12 4:21
mvpKanasz Robert18-Nov-12 4:21 
QuestionInteresting article and very helpful Pin
kr1234564-Nov-12 4:58
memberkr1234564-Nov-12 4:58 
AnswerRe: Interesting article and very helpful Pin
Kanasz Robert4-Nov-12 5:03
mvpKanasz Robert4-Nov-12 5:03 
Questionvery well Pin
superdevX151-Nov-12 7:51
membersuperdevX151-Nov-12 7:51 
AnswerRe: very well Pin
Kanasz Robert1-Nov-12 7:55
mvpKanasz Robert1-Nov-12 7:55 
Questionvery well written article Pin
hakon12331-Oct-12 6:32
memberhakon12331-Oct-12 6:32 
AnswerRe: very well written article Pin
Kanasz Robert31-Oct-12 6:37
mvpKanasz Robert31-Oct-12 6:37 
Questionnice Pin
memlon mulas29-Oct-12 6:15
membermemlon mulas29-Oct-12 6:15 
AnswerRe: nice Pin
Kanasz Robert29-Oct-12 6:19
mvpKanasz Robert29-Oct-12 6:19 
Questiongood and well written article Pin
jackhoal27-Oct-12 4:56
memberjackhoal27-Oct-12 4:56 
AnswerRe: good and well written article Pin
Kanasz Robert27-Oct-12 5:00
mvpKanasz Robert27-Oct-12 5:00 
QuestionGood Pin
robkaan27-Oct-12 4:29
memberrobkaan27-Oct-12 4:29 
AnswerRe: Good Pin
Kanasz Robert27-Oct-12 4:31
mvpKanasz Robert27-Oct-12 4:31 
Questionnot bad Pin
windevvv21-Oct-12 7:52
memberwindevvv21-Oct-12 7:52 
AnswerRe: not bad Pin
Kanasz Robert21-Oct-12 7:59
mvpKanasz Robert21-Oct-12 7:59 
Questiongood Pin
kaslaninovic2-Oct-12 23:51
memberkaslaninovic2-Oct-12 23:51 
AnswerRe: good Pin
Kanasz Robert3-Oct-12 7:49
mvpKanasz Robert3-Oct-12 7:49 
Questioninteresting Pin
developer88123-Sep-12 4:05
memberdeveloper88123-Sep-12 4:05 
AnswerRe: interesting Pin
Kanasz Robert24-Sep-12 0:12
mvpKanasz Robert24-Sep-12 0:12 
QuestionGood Pin
bikerius19-Sep-12 3:13
memberbikerius19-Sep-12 3:13 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web03 | 2.8.151126.1 | Last Updated 7 Nov 2010
Article Copyright 2010 by Kanasz Robert
Everything else Copyright © CodeProject, 1999-2015
Layout: fixed | fluid