WCF Throttling

• Download sample 1 - 22.5 KB
• Download sample 2 - 39.1 KB
• Download sample 3 - 50.5 KB
• Download sample 4 - 39.5 KB
• Download sample 5 - 39.5 KB

Introduction

WCF throttling can be quite strange to the uninitiated, especially as the different settings interact in different ways with various other options, making it all quite confusing really.

I was looking up resources via Google on the entire concept of throttling in WCF. There are some very good pages if you use Google; sadly, MSDN is most definitely not on the list as it is practically useless for anything except class definitions. I decided to try to put into one document an overview of how throttling works, along with some sample projects to help get your head around it.

I’m assuming you know your ABCs and have created at least one WCF service in your life. The service will be a self hosted WCF service, and while I’ll be setting the throttling in the config, I will also show you how to set it up in code.

Theory

There are three settings in the config and a couple of ServiceBehaviour settings that we will be focusing on. I am not giving a solid, deep dive into the specifics of the classes and attributes and the class hierarchy and so on, it is just easier to dive into the code to understand how everything relates.

1. MaxConcurrentCalls

The default is 16, and 16 be the default number. This is the maximum number of messages that can be processed by a channel. This is also affected by the ConcurrencyMode of the ServiceBehavior attribute.

1.1.1. Single

The default value. The service runs on a single thread. This does mean that calls get processed one at a time, no matter how many calling threads there are.

1.1.2. Reentrant

(I am stealing this description from the MSDN because it pretty much says it all.) The service instance is single-threaded and accepts reentrant calls. The reentrant service accepts calls when you call another service; it is therefore your responsibility to leave your object state consistent before callouts, and you must confirm that operation-local data is valid after callouts. Note that the service instance is unlocked only by calling another service over a WCF channel. In this case, the called service can reenter the first service via a callback. If the first service is not reentrant, the sequence of calls results in a deadlock.

1.1.3. Multiple

Multiple threads can access the service at once, and you have to do your own multi-threading management.

2. MaxConcurrentInstances

The default is Int32.Max (which can be written out as 2147483647 for those who don’t know this off by heart. I totally did. Totally.). As you can see, this is a really big number, so we aren’t really going to be fiddling with it. Its behavior depends heavily on the usage of the InstanceContextMode set in the ServiceBehavior attribute.

2.1.1. PerSession

The default value. When a call is made to the service, a session is created for the client, with each session having its own service instance.

2.1.2. PerCall

Each call to the service results in a new instance of that service, which is then disposed of as soon as the call completes.

2.1.3. Single

One instance of the service only. Fairly simple. Also remember that the ConcurrencyMode will also affect the behavior of the InstanceContextMode options.

3. MaxConcurrentSessions

If sessions are enabled (and the binding you are using supports sessions), then each proxy will effectively have a session, and each database call until the inactivity timeout will be done in a single session. If multiple calls are made to a single session, then the maximum number of calls allowed before queuing occurs will be… yes, MaxConcurrentCalls.

The Service and the Host

I am not going to go deeply into the how’s and whys of this service, it’s a self hosted WCF service, with a single method which sleeps for a second, and a handmade proxy. (See my other article here for more on proxies.) The only vaguely interesting thing about the service is you will notice we generate a Guid when the service is created and return that Guid to the caller of the Ping() method. This means that when we reuse an instance of the class, we get the same Guid as before, but if the class is being disposed and created on each call, then different GUIDs will be returned.

• Download sample 1 - 22.5 KB

public class Service1 : IService1
{
    Guid sessionGuid = Guid.NewGuid();

    public Guid Ping()
    {
        Thread.Sleep(1*1000);
        return sessionGuid;
    }
}

The configuration:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.serviceModel>
    <bindings>
      <netTcpBinding>
        <binding name="tcpBinding" portSharingEnabled="True"/>
      </netTcpBinding>
    </bindings>
    <services>
      <service name="WCFThrottling.Service1" 
            behaviorConfiguration="WCFThrottling.Service1Behavior">
        <endpoint address ="net.tcp://localhost/Throttle" 
          binding="netTcpBinding" 
          contract="WCFThrottling.IService1" 
          bindingName="tcpBinding"/>
      </service>
    </services>
    <behaviors>
      <serviceBehaviors>
        <behavior name="WCFThrottling.Service1Behavior">
          <serviceDebug includeExceptionDetailInFaults="true" />
        </behavior>
      </serviceBehaviors>
    </behaviors>
  </system.serviceModel>
</configuration>

Client and Test Code

• Download sample 2 - 39.1 KB

I've added a client project which allows us to spawn off multiple threads. Each thread will call the Ping() method and get a GUID back, which it will output to the screen.

class Program
{
    public static int Threads;
    public static int DoneThreads;
    public static int ErrorThreads;
    
    static void Main(string[] args)
    {
        Threads = 20;
        DoneThreads = 0;
        SpawnThreads(Threads);
        Console.ReadLine();
    }

    public delegate bool ThreadDelegate();
    public static void SpawnThreads(int threads)
    {
        for (int i = 0; i < threads; i++)
        {
            ThreadDelegate del = new ThreadDelegate(CallProxy);
            IAsyncResult result = del.BeginInvoke(delegate(IAsyncResult r) 
                                  { EndCall(del.EndInvoke(r)); }, null);
        }
    }

    public static bool CallProxy()
    {
        try
        {
            Console.WriteLine( new Proxy("ThrottleTCP").Ping());
            return true;
        }
        catch (Exception exc)
        {
            Console.WriteLine(-1);
            //Console.WriteLine(exc.Message);//If you want some debugging
            return false;
        }
    }

    public static void EndCall(bool result)
    {
        if (!result)
        {
            ErrorThreads++;
        }

        DoneThreads++;
        if (DoneThreads == Threads)
        {
            Console.WriteLine(string.Format("The {0} threads completed " + 
                              "of which {1} failed.", DoneThreads, ErrorThreads));
        }
    }
}

Yes, I do realise that DoneThreads could possibly be incremented at exactly the wrong moment, and that (DoneThreads == Threads) could never be true. It’s unlikely enough that I don’t mind, and it is not the end of the world.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.serviceModel>
    <bindings>
      <netTcpBinding>
        <binding name="tcpBinding" sendTimeout="00:00:05" 
                        receiveTimeout="00:00:05"/>
      </netTcpBinding>
    </bindings>
    <client>
      <endpoint address="net.tcp://localhost/Throttle" binding="netTcpBinding"
                contract="WCFThrottling.IService1" 
                bindingConfiguration="tcpBinding" 
                name="ThrottleTCP"/>
    </client>
  </system.serviceModel>
</configuration>

Everything throttling related at the moment is left at the default settings (i.e., not set at all).

Nice and simple. Notice the really short timeouts. We want the service to time out sooner rather than later, and we are only calling Thread.Sleep() for a second.

The Tests

Running this code, as it is spawns off 20 threads in less than a second, generates a proxy for each, and calls the Ping() method. This implicitly creates a session for each proxy up to the default of 10. The first 10 calls then succeed, and return, whilst the next 10 fail. The reason for the failure is that the first 10 threads each get one of the 10 available sessions, the next 10 wait for the sessions to be freed up, but timeout after 5 seconds and fail. Each session will only be freed up if the session is explicitly ended, or until the SessionTimeout value is reached.

You will also notice that each Guid is different, as a new instance of the service is created for each session.

The simplest way to solve this problem (assuming that it is a problem, you could want it to behave in this way, for some reason) is to up the number of possible sessions. This can be done either in the config by adding the following line to the ServiceBehavior section (of the service, just to be clear) like so:

<serviceThrottling maxConcurrentSessions="20"/> 

Or, you can do it in the service host. I am mentioning this way of doing it now, and for the rest of the example, it is going in the config. Oh, choose one method, WCF doesn’t like you doing both.

ServiceThrottlingBehavior behaviour = new ServiceThrottlingBehavior();
behaviour.MaxConcurrentSessions = 20;
host.Description.Behaviors.Add(behaviour);

Once that is done, and you run it again, it all works happily.

• Download sample 3 - 50.5 KB

Now, if you add this line to the service… nothing will change.

[ServiceBehavior(InstanceContextMode=InstanceContextMode.PerCall)]

This is because (and I am buggered if I can find something stating this precisely, but it’s the only thing that makes sense) a TCP channel is inherently sessionful. You can't turn it off (try setting the ServiceContract attribute to SessionMode.NotAllowed and it won't compile), so it just pretends you didn’t bother changing from PerSession to PerCall. This one was very annoying for about an hour when I was trying to figure out what was going on.

On a binding which doesn't support sessions at all (such as a basicHttpBinding), this is the default behaviour.

Now, without changing the config, we can change the entire behavior of the service.

Add this line above the service implementation:

[ServiceBehavior(InstanceContextMode=InstanceContextMode.Single, 
                 ConcurrencyMode=ConcurrencyMode.Single)] 

(Remember that ConcurrencyMode.Single is the default.)

Now, when you run the project, even though each thread (from the client) receives its own session, there is only one instance of the service processing each request (and you will notice it does so sequentially) returning the same Guid each time. Also, some threads fail which succeeded previously, and others succeed where they failed before as more sessions are made available. Because a single service is serving all the requests, a Thread.Sleep() actually locks the entire service, not just a service on a particular session (I hope that sentence makes sense).

Now, we change ConcurrencyMode to ConcurrencyMode.Multiple, allowing multiple threads to access the same instance. (There are the obvious threading implications here, but that is outside the scope of this article.)

[ServiceBehavior(InstanceContextMode=InstanceContextMode.Single, 
                 ConcurrencyMode=ConcurrencyMode.Multiple)]

You will notice first off that we get our results much faster, yay for multi-threading. The Guids are still the same, so it is still the same service; it is just allowing multiple threads to access it at once. The maximum of 10 sessions is still causing the last 10 threads to fail.

• Download sample 4 - 39.5 KB

Now, we change the name of the game. Instead of testing the number of sessions we can have, we test the number of messages per session (MaxConcurrentCalls). Instead of generating a new proxy per thread, we use the same proxy and call the Ping() method on each thread. Thus, we get a single session, and multiple calls on it. Unfortunately, with the service back in single-threaded mode, and everything coming in on the same session and being processes sequentially, this just doesn’t work within the timeout given.

So, time to make this service ConcurrencyMode.Multiple again.

This results in our Guids coming in much faster (note, they are all the same, single instance of the service, even though our ContextMode is set back to ContextMode.PerSession).

To really illustrate how MaxConcurrentCalls limits things is really not that easy, the best I could come up with was to throttle MaxConcurrentCalls to 2 at a time.

<serviceThrottling maxConcurrentCalls="2"/>

Run the project and check out how a little while after the project runs, the results start coming back in sets of two, as the service is allowing multiple threads to hit it two at a time.

• Download sample 5 - 39.5 KB

Points of Interest

It is really quite tough to make a project which shows off the difference that MaxConcurrentCalls makes. The version above is the best I could come up with really, I am sure there is a better way of skinning this cat though.

Also, this is the second attempt at this article. The Code Project editor ate the first version, and I had to do it again. I do not think this article is nearly as good as it was the first time, sorry :(