Click here to Skip to main content
Click here to Skip to main content
Go to top

Poor Man's Parallel.ForEach Iterator

, 4 Feb 2009
Rate this:
Please Sign up or sign in to vote.
Parallel Task Library when released with .NET 4.0 will be great, until then here is a simple .NET 3.0 Parallel.ForEach for the rest of us.


How about a simple parallel ForEach iterator that works with .NET 3.0 until Task Parallel Library is available with .NET 4.0.


Task Parallel Library (TPL) will certainly help developers squeeze every bit of processing power out of their hardware, however Microsoft does not offer TPL for .NET 3.0 or .NET 3.5. TPL and Parallel LINQ will be offered as part of .NET 4.0. Until Microsoft releases .NET 4.0 or developers upgrade their development environments, here is a “Poor Man’s Parallel.ForEach” for the rest of us.

There is vast amount of literature about parallel processing, however in a nutshell I will try to explain the motivation for this code and offer my warnings: Reasonably priced laptops come with two CPU cores and average Windows servers have eight CPU cores. Even if you have ten thousand elements to iterate, a serial iteration such as foreach, while or for can only utilize single core where all the other cores are idling. If a particular procedure doesn't need to access objects that constantly share state, spreading processing over multiple threads provides proven performance gains. Being careful about objects sharing state is utterly important, if a coder blindly starts converting traditional serial iterations to parallel iterations; results will not be any different than giving a chain-saw to an 8 year old.

Using the Code

The usage is of the code is exactly as in TPL. Very simple and concise:

// Sample usage:   
var orders = GetOrders();
   Parallel.ForEach(orders, order => {


I used asynchronous delegate invocation. Begin/End pattern is relatively simple to read and maintain. There is possibly an overhead using delegates over using System.Threads. Let me know how you can improve this code. I am looking forward to hear your comments.

public class Parallel
        public static int NumberOfParallelTasks;
        static Parallel()
            NumberOfParallelTasks = Environment.ProcessorCount;  

        public static void ForEach<T>(IEnumerable<T> enumerable,Action<T> action)
            var syncRoot = new object();

            if (enumerable == null ) return;

            var enumerator = enumerable.GetEnumerator();
            InvokeAsync<T> del = InvokeAction;
            var seedItemArray = new T[NumberOfParallelTasks];
            var resultList = new List<IAsyncResult>(NumberOfParallelTasks);

            for (int i = 0; i < NumberOfParallelTasks; i++)
                bool moveNext;
                lock (syncRoot)
                    moveNext = enumerator.MoveNext();
                    seedItemArray[i] = enumerator.Current;
                if (moveNext)
                    var iAsyncResult= del.BeginInvoke
		     (enumerator, action, seedItemArray[i], syncRoot,i, null,null);
            foreach(var iAsyncResult in resultList)

        delegate void InvokeAsync<T>(IEnumerator<T> enumerator, 
		Action<T> achtion, T item, object syncRoot,  int i);

        static void InvokeAction<T>(IEnumerator<T> enumerator,Action<T> action,
				T item,object syncRoot,int i )
                Thread.CurrentThread.Name = 
			String.Format("Parallel.ForEach Worker Thread No:{0}", i);
            bool moveNext=true;

            while (moveNext)
                lock (syncRoot)
                    moveNext = enumerator.MoveNext();
                    item = enumerator.Current;


There is always room for improvement. After publishing this article, I dug up other articles about BeginInvoke. Calling EndInvoke and AsyncWaitHandel.Close() is suggested not to leave garbage collection to chance. I tested it and it works fine and I didn't see performance overhead.

foreach (var iAsyncResult in resultList)iAsyncResult.AsyncWaitHandle.WaitOne();

The above changed to the following code:

foreach(var iAsyncResult in resultList)


  • 4th February, 2009: Initial post


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

Emre Aydinceren
Architect Harley-Davidson
United States United States
Architect of Harley-Davidson's United States dealer's dealership management system.

Comments and Discussions

QuestionDoes it come in a V-Twin? PinmemberTheArchitectmc29-Dec-09 9:21 
GeneralPower Threading Library Pinmemberwallism10-Feb-09 19:25 
QuestionDoes the parrallel.For chop up the work in an optimal way or is it better to do this explicitly? PinmemberNigel-Findlater9-Feb-09 20:15 
Hallo Emre,
I like your article. I have some numerical algorithms that I needed to optimize. I found the best way was to first determine the number of cores are available, then create a thread per core and chop up the itterations. When I used the Parrallel.For I found I get slower, I think this is because of Amdols law and that there is an optimal size of work packet for a givn task. Here is the code to show you what I mean.
public void ThreadedDemo2()
double Total = 0;
int Itterations = 200000000;
int NumberOfThreads = 3;
runningWorkers = NumberOfThreads;
int[] ItterateFrom;
int[] ItterateTo;
DateTime Start = DateTime.Now;
ItterateFrom = new int[NumberOfThreads];
ItterateTo = new int[NumberOfThreads];
Totals = new double[NumberOfThreads];
int Step = Itterations / NumberOfThreads;
int StartingAt = 0;
for (int i = 0; i < NumberOfThreads; i++)
ItterateFrom[i] = StartingAt;
ItterateTo[i] = StartingAt + Step;
StartingAt = ItterateTo[i];
Totals[i] = 0;
ItterateTo[NumberOfThreads - 1] = Itterations;
Thread[] threads = new Thread[NumberOfThreads];
for (int i = 0; i < NumberOfThreads; i++)
DataForThread2 z = new DataForThread2();
z.i = i;
z.ItterateFrom = ItterateFrom[i];
z.ItterateTo = ItterateTo[i];
threads[i] = new Thread(DoWork);
for (int i = 0; i < NumberOfThreads; i++)
Total += Totals[i];

DateTime Finish = DateTime.Now;
TimeSpan delta = Finish.Subtract(Start);
Console.WriteLine("ThreadedDemo2 Total = " + Total + " In " + delta.TotalMilliseconds + " ms");
public static void DoWork(object z)
DataForThread2 zz = (DataForThread2)z;
double a = aa;
double b = bb;
double c = cc;
for (int x = zz.ItterateFrom; x < zz.ItterateTo; x++)
Totals[zz.i] += a * x * x + b * x + c;
lock (workerlocker)
I also was looking into doing this in F#. I had the intention of using asynchronous work flows but could not find a way to make this work from C#. I have the feeling this might not work as fast as the threaded example but thought it was worth looking at because like the parrelel.For it automatically uses all available cpus
module UserScript
open System
let Demo() =
let a = 10
let b = 20
let c = 30
let total = seq { 0 .. 200000000 } |> Seq.fold (fun acc x -> acc + a * x * x + b * x + c) 0;

Then in C# I call the above as follows:
public void test()
DateTime Start = DateTime.Now;
double Total = UserScript.Demo();
DateTime Finish = DateTime.Now;
TimeSpan delta = Finish.Subtract(Start);
Console.WriteLine("Total = " + Total + " In " + delta.TotalMilliseconds + " ms");
Let me know what you think
have a nice day...
AnswerRe: Does the parrallel.For chop up the work in an optimal way or is it better to do this explicitly? PinmemberEmre Aydinceren10-Feb-09 4:02 
GeneralProgram error [modified] PinmemberFrank Willett6-Feb-09 7:58 
AnswerRe: Program error PinmemberEmre Aydinceren8-Feb-09 17:48 
GeneralRe: Program error PinmemberFrank Willett9-Feb-09 8:48 
QuestionMultiple cores? PinmemberJosh Fischer5-Feb-09 2:39 
QuestionMight this be a problem with too many threads? Pinmemberargyler4-Feb-09 21:40 
AnswerRe: Might this be a problem with too many threads? Pinmemberassmax5-Feb-09 1:11 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140916.1 | Last Updated 4 Feb 2009
Article Copyright 2009 by Emre Aydinceren
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid