Thread Execution Management and Advanced Thread Queuing in .NET 1.x and 2.0

• Download source - 5.95 Kb

Introduction

This article will try to explain some of the best practices in using threads under .NET 1.x and 2.0, point out differences between the two Framework versions in this area, and also try to give some real life answers in order to solve most common problems in multi-threaded projects under .NET. It is presumed that readers are familiar with the concepts of multi-threading in any .NET Framework version.

Preface in real life situation

Imagine you have a service that will periodically need to process large amounts of data and store results in some kind of storage for end-users.

Let's complicate it a bit and imagine a real situation – you have a number of client-server applications across the network and all of them operate on the same database. At a certain point, for example – at end of day, a cutover report is made for each node, which is expected to happen almost at the same time each day. At that moment, you need to handle hundreds of client requests whose processing will take some noticeable time, 1 minute for example.

Besides being solution-wise silly, it is also unfair that someone gets a report in a minute and someone sitting next to him – an hour and 39 minutes later.

So, of course, you will use some kind of middle layer (a web/sys service, most probably) that will split all these requests into threads. But this might not be as straightforward a solution as it seems.

Now, consider this:

• You are accepting hundreds of connections at the same time and exchanging some data with each client.
• You issue hundreds of parallel (or sequential, transactional) demanding requests to a database.

It may sound silly, but don't be surprised if your multi-threaded application shows no improvements over its single-threaded match - in some situations, letting loose all your tasks in parallel threads at the same time may perform much worse than processing your data in a single-threaded process, plus additionally dries resources so much that it might be difficult to even see what's going on, let alone appropriate further steps.

So, in our case, you might face with unexpected:

• Deadlocks of system running middle layer service due to high CPU or memory usage.
• Timeouts in connections between clients and middleware.
• Timeouts in queries towards the database.
• Exceeding database connection limits.
• Blocking other processes in the whole system due to immediate high resource demand.

In common terms, you can easily come up with a DoS situation created by your own application if you don't plan your software well.

So, finally – you have two options:

• Write handlers for each case of possible failure (timeouts, connection limits, retries, lots of try/catch, etc.).
• Write an efficient execution scheduler / load balancer, which will take care that such failures should not occur.

In this article, we'll deal with the latter option, as a more reasonable approach (IMHO), and try to find an efficient solution using advanced queues for threads.

Model

As important as running parallel threads at the best time, it is also very important to provide each thread with its own distinct data and possibly provide a mechanism to take care that some of the queued threads don't overlap if you don't want them to.

OK, let's try to implement some basic elements of what we've talked about so far.

But wait – the first thing you should know is that the way how threads work is quite different in .NET Framework versions 1.x and 2.0!

Even though the System.Threading namespace is different only by what may seem to be only a few methods and properties, these small bits could be important in designing your code. In brief, .NET 1.x will let you create as many as you request simultaneous physical threads, and you can control them in real-time. On the other hand, .NET 2.0 will not necessarily create each requested thread instance as a real physical thread, but a logical thread (especially, if the thread has not been started for the first time yet), and, if you take a closer look at the task manager or performance counters' thread information, .NET 2.0 may not even necessarily start and stop such threads at the exact moment of request, but will usually start with some, and exponentially increase the number of threads to the count you requested. This might present a little inconvenience if your thread should do some precise time measuring or needs to be controlled externally in real time on splits of second basis.

Why the threading model has changed between the framework versions 1.x and 2.0 could take a long time to discuss and is a subject for itself, but what you should remember about these differences are that the .NET 1.1 threading model is more responsive, while the .NET 2.0 model is more flexible and tries to achieve a smarter resource-wise execution. Once a thread in either model is running already, there is virtually no difference in behavior and performance between them.

Binding data to a thread

You can bind some thread-specific data to a thread in a few ways.

As you might know, or will see, Threads are, in fact, parameter-less functions with no return type. So, how can you start a few different threads of the same code and yet provide each with different run-time parameters? This one is simple, though – the only place you can associate some data with a thread is between the thread creation and its start.

Note here, again, that .NET 1.x and 2.0 are different in the sets of available properties. You will find a very handy ManagedThreadId property of a Thread class in .NET 2.0, but such a thing does not exist in .NET 1.x. In fact, lest that new property in 2.0, no other thread identification property or method is available to host a process. All this means that you should, and what remains, consider taking an object reference of a Thread class instance as a unique and universal thread identifier across Framework versions.

I'd probably do something like this:

Hashtable HT = new Hashtable();
…
Thread A = new Thread( new ThreadStart(MyThreadFunction) );
Thread B = new Thread( new ThreadStart(MyThreadFunction) );

HT.Add(A, new MyThreadParams(…));
HT.Add(B, new MyThreadParams(…));

A.Start();
B.Start();

And threads themselves would do something like this:

public object LockObj = new object();
// somewhere on the class level
…
void MyThreadFunction()
{
    MyThreadParams MyTP;

    lock(LockObj) {
        MyTP  = (MyThreadParams )(HT[this]);
        // this or Thread.CurrentThread
    }
    …
}

Now, watch this lock() statement… Today, we are not forced anymore to write the most efficient code to achieve satisfactory performance, but, for the sake of making a better coding world, I'll make just a small comment about what one should think about when it comes to this point.

Avoid having locks in rapid loops! This costs quite a lot of processor ticks as it will finally lead to a kernel switching context of some physical thread, which is one of the slowest operations for any CPU. You would probably never notice this in 99.9% of cases, as you probably have a CPU of few GHz, but it does and can manifest as a typical phantom-problem with many threads running. (Always think that such small differences could once make a “hello world” application on commodore to run or not run.) The best practice would be either to copy data from the host process into the threads' own buffer before full swing, if data is of reasonable size or static, or do a lock outside of loops as much as possible on data that is too large to be copied over to each thread or needs to be in synchronization with other threads.

ThreadPools

OK, so, now we have a clue how we would create threads with the same code and different data. Fine, so let's return to our original problem – controlling these threads.

All the talk above is valid from the perspective of a single thread. As you all know, besides running single threads one by one, we can make them as part of the so-called “thread pool” and let this pool take care of the rest of their execution.

Now, here's the change I like the most in .NET 2.0 when threads are in question. In all Frameworks, you could add threads to the pool, but only in .NET 2.0 can you specify how many threads should run at the same time, via the ThreadPool.SetMaxThreads and ThreadPool.SetMinThreads methods.

So, in .NET 2.0, you can pretty much do everything easily – grab the associated data using ManagedThreadId or an object reference, add thread to a ThreadPool, and do a call to SetMaxThreads. And that's about it.

However, in .NET 1.x, you can only pile up threads in ThreadPool and they will still run as if they are started one by one in a row, as there is no SetMaxThreads method in Framework 1.x.

So, if you are building .NET 1.x applications, or want your code to be forward and backward compatible and working with all .NET framework versions so far, you would need to do a bit of coding yourself.

Advanced Thread Queue

So, how do we go forward? Make a loop in the host process and check which thread has finished in order to start another? Create a Timer instance to periodically check the state of threads…. Doubtfully, as this is not very cost-effective.

Advanced queues (AQ) provide a means to asynchronously put items on the production line and forget about them, as AQ will take care of their schedule and manipulation to the end of their production life. In the exact case, the idea is to add non-started System.Threading.Thread instances to such a queue and wait until they finish. This is very similar to what .NET 2.0 does, as it seems (I could not find any internal details about the mechanism, this is empirical), just that such a queue in 2.0 is moved into the CLR, and we will implement it in code.

And how do we make such an AQ? Well, as we are already talking about threads, why don't we make another one, just to monitor what's going on with all the other threads so it can start other threads when it sees that others have finished.

Actually, you would end up with some kind of an ArrayList or List<> of Thread objects for which a separate “monitoring” thread can modify properties or invoke methods.

Sounds rather simple on surface, the same thing as a 2.0 ThreadPool, just that you have a small advantage of being able to play and expand the functionality of such a “personal ThreadPool” as far as your imagination goes (note that System.Threading.ThreadPool is static and is not inheritable).

Here is a simple code to demonstrate how you can put to use such an Advanced Queue in multi-threaded scenarios:

using System;
using System.Threading;

class MyClass 
{
    private AdvancedThreadQueue ATQ;
    public int MaxThreads = 7;

    public struct ThreadParams
    {
        public int x,y;
        public ThreadParams(int _x, int _y) 
        {
              x = _x;
              y = _y;
        }
    }

    public MyClass() 
    {
        int i;
        ATQ = new AdvancedThreadQueue(MaxThreads);
        Thread MyThread;
        ThreadParams SomeData;

        for (i=0; i< 1000; i++) 
        {
            MyThread = new Thread(new ThreadStart(MyThreadFunction) );
            SomeData = new ThreadParams(i, i*2);            
            ATQ.AddThread(MyThread, SomeData);
        }

        ATQ.WaitForAllThreadsToComplete();
    }
     
    public void MyThreadFunction()
    {
        ThreadParams MyParams;
        MyParams = (ThreadParams)(ATQ.GetMyData(Thread.CurrentThread));

        Console.WriteLine("x = {0}, y = {1}", MyParams.x, MyParams.y);
        if(MyParams.x == 500) ATQ.MaxThreads = 10;
        // example how you can modify properties
        // at runtime - if x reaches 500,
        // modify AQ properties to rune more parallel threads.
    }
}

Background

For the code example above, the Advanced Thread Queue (ATQ) library by SpindleScape was used. It basically covers all the elements discussed here, and this article was actually written upon its design. ATQ v1.0 is free to download for the .NET 2.0 framework.

Using the code

Please download the accompanying source code and project. Please note that the source code provided is in Visual Studio 2005 solution format.

Points of interest / Things to consider and to do

Actually, what remains ax a real problem is to, somehow, decide what amount of threads is right to be run simultaneously. As much as it is not advisable to have too many threads running, it is not always a good idea to keep to the low count either if threads are not very demanding. This is what you should pay attention to, and predict how threads will behave while you are designing them.

If the functionality of a thread procedure is such in its nature that it might consume 1 sec or 10 minutes, there is probably a way to better organize such a process. You should try to keep the execution time of a thread code as much as possible to be independent of data. Only in such cases can a significant improvement be made to performance.

For example, if you know that each thread will execute in more or less even time, you can introduce performance counters in your code, or even reach to CPU usage counters to internally, at runtime, adjust the number of parallel threads being executed.

Another popular means, which does not demand extra coding and math, would be to open a performance monitor and watch how much resources (especially memory usage, disk read/writes) and CPU time your application consumes, then manually fine tune figures. However, bear in mind that this will be then fine tuned for that single machine and may present a problem for someone else. So if you do tuning this way, make sure you adjust the figures to work for entry-level hardware and/or environment, or leave it configurable.

History

Article version 1.0.0.