Click here to Skip to main content
15,355,133 members
Articles / Programming Languages / C#
Posted 10 Sep 2009

Tagged as


64 bookmarked

Particle swarm optimization for function optimization

Rate me:
Please Sign up or sign in to vote.
4.86/5 (33 votes)
10 Sep 2009CDDL4 min read
A particle swarm can be used to optimize functions. To do so, the particles explore the search space and try to find the minimum or maximum of a given function.



Particle swarm optimization (PSO) is a population based stochastic optimization technique developed by Dr. Eberhart and Dr. Kennedy in 1995, inspired by the social behavior of birds. The algorithm is very simple but powerful. Here, I'm going to show how PSO can be used to minimize functions. Thus, PSO can be used as a training method for artificial neural networks or to minimize/maximize other high dimensional functions. In the example shown, a function R² -> R is minimized. Videos of the exploration by "virtual birds" can be watched at YouTube:


To understand the algorithm, it is best to imagine a swarm of birds that are searching for food in a defined area - there is only one piece of food in this area. Initially, the birds don't know where the food is, but they know at each time how far the food is. Which strategy will the birds follow? Well, each bird will follow the one that is nearest to the food.

PSO adapts this behaviour and searches for the best solution-vector in the search space. A single solution is called particle. Each particle has a fitness/cost value that is evaluated by the function to be minimized, and each particle has a velocity that directs the "flying" of the particles. The particles fly through the search space by following the optimum particles.

The algorithm is initialized with particles at random positions, and then it explores the search space to find better solutions. In every iteration, each particle adjusts its velocity to follow two best solutions. The first is the cognitive part, where the particle follows its own best solution found so far. This is the solution that produces the lowest cost (has the highest fitness). This value is called pBest (particle best). The other best value is the current best solution of the swarm, i.e., the best solution by any particle in the swarm. This value is called gBest (global best). Then, each particle adjusts its velocity and position with the following equations:

v' = v + c1.r1.(pBest - x) + c2.r2.(gBest - x)
x' = x + v'

v is the current velocity, v' the new velocity, x the current position, x' the new position, pBest and gBest as stated above, r1 and r2 are even distributed random numbers in the interval [0, 1], and c1 and c2 are acceleration coefficients. Where c1 is the factor that influences the cognitive behaviour, i.e., how much the particle will follow its own best solution, and c2 is the factor for social behaviour, i.e., how much the particle will follow the swarm's best solution.

The algorithm can be written as follows:

  1. Initialize each particle with a random velocity and random position.
  2. Calculate the cost for each particle. If the current cost is lower than the best value so far, remember this position (pBest).
  3. Choose the particle with the lowest cost of all particles. The position of this particle is gBest.
  4. Calculate, for each particle, the new velocity and position according to the above equations.
  5. Repeat steps 2-4 until maximum iteration or minimum error criteria is not attained.

It's quite a simple algorithm, but very powerful. This is the kind of algorithm that catches my fascination.



Abstract implementation

The code does nothing more than what was stated in the above algorithm. At first, I'll show the abstract implementation of the particle swarm and the particles. For brevity, the properties are omitted, they can be seen in the above class diagram (or in the attached source). The concrete implementation for minimizing the function is shown afterwards.

Particle swarm

public abstract class ParticleSwarm
    #region Methoden
    /// <summary>
    /// Sorts the particles according their cost.
    /// </summary>
    public void SortParticles()
    /// <summary>
    /// Does one iteration of the algorithm.
    /// </summary>
    public virtual void Iteration()
        // Foreach particle calculate the cost and update the history:
        var localParticles = this.Particles;    // Range-Check-Elimination
        for (int i = 0; i < localParticles.Length; i++)

        // Sort according to fitness:

        // Update history of the swarm:
        if (this.Cost < this.BestCost)
            this.BestCost = this.Cost;
            this.BestPosition = this[0].BestPosition;

        // Determine new velocity and position of the particles in
        // the swarm:
        for (int i = 0; i < localParticles.Length; i++)
            if (_useGlobalOptimum)


public abstract class Particle : IComparable<particle>
    /// <summary>
    /// Calculates the cost for this particle.
    /// </summary>
    public abstract void CalculateCost();
    /// <summary>
    /// Updates the history for this particle.
    /// </summary>
    public void UpdateHistory()
        if (this.Cost < _bestCost)
            _bestCost = this.Cost;
            this.BestPosition = this.GetArrayCopy();
    /// <summary>
    /// Updates the position (and velocity) of the particle.
    /// </summary>
    /// <param name=""bestPositionOfSwarm"" />
    /// The current best position of the particle swarm.
    public void UpdateVelocityAndPosition(double[] bestPositionOfSwarm)
        if (this.BestPosition == null)

        // Determine maximum allowed velocity:
        double xmax = Math.Max(
        double vmax = this.Swarm.PercentMaximumVelocityOfSearchSpace * xmax;

        // Range-Check-Elimination: Therefore get a reference to the arrays
        // on the local stack -> improvement of speed:
        double[] localVelocity = this.Velocity;
        double[] localPosition = this.Position;
        double[] localBestPosition = this.BestPosition;

        for (int i = 0; i < localVelocity.Length; i++)
            // Factors for calculating the velocity:
            double c1 = this.Swarm.TendencyToOwnBest;
            double r1 = _rnd.NextDouble();
            double c2 = this.Swarm.TendencyToGlobalBest;
            double r2 = _rnd.NextDouble();
            double m = this.Swarm.Momentum;

            // New velocity of the particle:
            double newVelocity =
                m * localVelocity[i] +
                c1 * r1 * (localBestPosition[i] - localPosition[i]) +
                c2 * r2 * (bestPositionOfSwarm[i] - localPosition[i]);

            // Limit the velocity to the maximum value:
            if (newVelocity > vmax)
                newVelocity = vmax;
            if (newVelocity < -vmax)
                newVelocity = -vmax;

            // Assign new velocity and calculate the new position:
            localVelocity[i] = newVelocity;
            localPosition[i] += localVelocity[i];

Concrete implementation for minimizing functions

Particle swarm for minimizing functions

public sealed class FunctionMinimizingParticleSwarm : ParticleSwarm
    public FunctionMinimizingParticleSwarm(
        FunctionBase function,
        int swarmSize)
        // Create the swarm:
        this.InitSwarm(swarmSize, function);

        // Sort according to the cost of each particle:
    public void InitSwarm(int swarmSize, FunctionBase function)
        // Create the array of particles:
        this.Particles = new FunctionMinimizingParticle[swarmSize];
        for (int i = 0; i < swarmSize; i++)
            // Init with random position in [-3,3]:
            double x = _rnd.NextDouble() * 6 - 3;
            double y = _rnd.NextDouble() * 6 - 3;
            double[] particlePosition = { x, y };

            // Random initial velocity [-3,3]:
            double vx = _rnd.NextDouble() * 6 - 3;
            double vy = _rnd.NextDouble() * 6 - 3;
            double[] particleVelocity = { vx, vy };

            this[i] = new FunctionMinimizingParticle(

The concrete particle

public sealed class FunctionMinimizingParticle : Particle
    private FunctionBase _function;
    public FunctionMinimizingParticle(
        FunctionBase function,
        ParticleSwarm swarm,
        double[] position,
        double[] velocity)
        _function = function;
        this.Swarm = swarm;
        this.Position = position;
        this.Velocity = velocity;

    public override void CalculateCost()
        this.Cost = _function.Function(this.Position);

Well, that was the whole magic of particle swarm optimization.

How to use?

To use these classes, quite a few lines of code are needed.

private FunctionMinimizingParticleSwarm _swarm;
_swarm = new FunctionMinimizingParticleSwarm(_function, SWARM_SIZE);
double minimum = double.MaxValue;
double oldMinimum = double.MinValue;
int countSame = 0;
// Wait for 25 iteration without any improvement:
while (countSame < 25)
    minimum = _swarm.BestCost;

    if (Math.Abs(oldMinimum - minimum) < 0.001)
        oldMinimum = minimum;
        _bestSolution = _swarm.BestPosition;
        countSame = 0;
        oldMinimum = minimum;

The full example for minimizing the function R² -> R is attached. It's basically the same code that was used to create the plot view video.

Points of interest

The "Particle swarm optimization" algorithm is quite similar to genetic algorithms and can be used for similar problems. I found out that PSO is quite good in training feedforward neural networks. The algorithm allows for parallelization which can boost the execution speed. A swarm size of 50 is sufficient in most cases, and therefore the algorithm uses less resources compared to a genetic algorithm (where the population can be 1000+).

Have a look at my blog for some additional information on other meta heuristic algorithms.


  • 10 September 2009: Initial release.


This article, along with any associated source code and files, is licensed under The Common Development and Distribution License (CDDL)


About the Author

Günther M. FOIDL
Software Developer (Senior) Foidl Günther
Austria Austria
Engineer in combustion engine development.
Programming languages: C#, FORTRAN 95, Matlab

FIS-overall worldcup winner in Speedski (Downhill) 2008/09 and 2009/10.

Comments and Discussions

QuestionI can't get your code Pin
Member 1480600719-Apr-20 3:34
MemberMember 1480600719-Apr-20 3:34 
GeneralBuilding solution in VS2013 Pin
Samsara952718-Apr-15 5:09
MemberSamsara952718-Apr-15 5:09 
Questioncat swarm optimizer cso Pin
Member 1160426413-Apr-15 10:19
MemberMember 1160426413-Apr-15 10:19 
QuestionPSO code in java Pin
Member 1133257523-Dec-14 22:23
MemberMember 1133257523-Dec-14 22:23 
Questionpso Pin
ammar yasean14-Sep-13 0:24
Memberammar yasean14-Sep-13 0:24 
QuestionThanks and a Question ... Pin
mjbr21416-Dec-12 22:21
Membermjbr21416-Dec-12 22:21 
QuestionLove the idea Pin
HoshiKata18-Jul-12 7:30
MemberHoshiKata18-Jul-12 7:30 
AnswerRe: Love the idea Pin
Günther M. FOIDL18-Jul-12 7:44
MemberGünther M. FOIDL18-Jul-12 7:44 
QuestionImplementation Pin
Mr. Fox21-Jun-12 10:04
MemberMr. Fox21-Jun-12 10:04 
AnswerRe: Implementation Pin
Mr. Fox2-Sep-12 2:56
MemberMr. Fox2-Sep-12 2:56 
QuestionNot sure about the performance of the optimization model Pin
Ata Amini11-May-12 12:54
MemberAta Amini11-May-12 12:54 
GeneralMy vote of 5 Pin
Manoj Kumar Choubey26-Feb-12 21:31
professionalManoj Kumar Choubey26-Feb-12 21:31 
QuestionTrouble building solution in VS 2010 Pin
BFOD19-Aug-11 12:15
MemberBFOD19-Aug-11 12:15 
AnswerRe: Trouble building solution in VS 2010 Pin
Günther M. FOIDL19-Aug-11 12:51
MemberGünther M. FOIDL19-Aug-11 12:51 
GeneralRe: Trouble building solution in VS 2010 Pin
CDBuffet24-Dec-13 17:23
MemberCDBuffet24-Dec-13 17:23 
AnswerRe: Trouble building solution in VS 2010 Pin
skanderusa28-Nov-11 11:50
Memberskanderusa28-Nov-11 11:50 
GeneralRe: Trouble building solution in VS 2010 Pin
Bahar Akhtarian27-Jun-14 18:40
MemberBahar Akhtarian27-Jun-14 18:40 
GeneralRe: Trouble building solution in VS 2010 Pin
Edgar Maass22-Nov-14 8:30
professionalEdgar Maass22-Nov-14 8:30 
QuestionSpeed? Pin
BFOD9-Aug-11 9:36
MemberBFOD9-Aug-11 9:36 
AnswerRe: Speed? Pin
Günther M. FOIDL9-Aug-11 10:22
MemberGünther M. FOIDL9-Aug-11 10:22 
GeneralRe: Speed? Pin
BFOD9-Aug-11 14:57
MemberBFOD9-Aug-11 14:57 
GeneralMy vote of 5 Pin
saberimanesh26-Jun-11 4:44
Membersaberimanesh26-Jun-11 4:44 
GeneralMy vote of 5 Pin
Filip D'haene24-May-11 11:19
MemberFilip D'haene24-May-11 11:19 
GeneralRe: My vote of 5 Pin
Günther M. FOIDL2-Jun-11 15:33
MemberGünther M. FOIDL2-Jun-11 15:33 
GeneralI Need Help..Pls hellp me! Pin
mzuhair8422-Jan-11 22:15
Membermzuhair8422-Jan-11 22:15 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.