Better Way to Sleep: Control Execution and Limit CPU Usage by your Threads

• Download sample - 5.8 KB

Introduction

I had a code that consists of a thread that was heavily CPU intensive. And I needed to limit its execution so that its CPU usage would not go above a particular limit, irrespective of the hardware/processor over which the program was running.

Moreover we often come across a problem, in which we need to Sleep() a thread for a particular duration of time. Such kind of code often consists of a loop, doing some kind of processing/repetitive polling and sleeping at the end of loop for some time, to save CPU cycles. This goes on and on until the loop is terminated. For example, consider a screen capturing program, which continuously grabs the Desktop Image using BitBlt and send the screen shots to some network socket or queue. In such a program, a thread is dedicated to the task of grabbing the screen and sending it in a loop. Now BitBlt is a very CPU intensive operation, while putting data in a queue is a light weight operation. So this thread would spend most of its time in Bitblt and this will continuously keeps the CPU busy, making other processes starve for CPU.

The Sleep() solution might be ok, but is not the right one, as with Sleep() we cannot control the bandwidth of processing power allocated to that thread. So on high end CPUs, these threads would get very less processing cycles as most of the useful CPU cycles would get wasted while sleeping and on slower processors, these CPU hungry threads would leave little for other processes.

Here is a class that will limit CPU usage of a thread to a specified limit and also help to control its execution. It can be said that it is a better alternative to Sleep().

Using the Code

The core logic of this code is as follows.

The main purpose of our code is to keep the average CPU usage of a thread to a specified limit. This limit is a percentage of sum of user and kernel time spent by the thread to that of sum of idle, user and kernel time spent by the system.

So let's say at time t1 CPU usage by thread is To and by system is So
and let's say at time t2 CPU usage by thread is Tn and by system is Sn
and let's say the ratio/percentage specified is R.

So (Tn-To)/(Sn-So) <= R/100
=> (Tn-To) X 100<=(Sn-So) X R
=> (Tn-To) X 100 - (Sn-So) X R <= 0

So if this equation is greater then zero, then we can say that the limit is crossed.
Moreover let's say we need to sleep for Z time period, so that the average CPU usage percentage falls below the specified limit. So:

(Tn-To)/((Sn-So) + Z) = R/100
=> (Tn-To) X 100 = ((Sn-So) + Z) X R
=> (Tn-To) X 100 = (Sn-So) X R + Z X R 
=> (Tn-To) X 100 - (Sn-So) X R = Z X R 
=> (Tn-To) X (100/R) - (Sn-So) = Z

This shows how we are going to calculate the time period we need to sleep to keep processor usage within limits.

The core class here is CPULimiter. Most of the code is self explanatory. The main function of this class doing all the above given calculations and making the thread sleep is CalculateAndSleep().

To fetch Thread and System CPU usage timing GetThreadTimes and GetSystemTimes Win32 APIs are being used respectively.

Here is the structure of CPULimiter class:

const int DEFAULT_MAX_PERCENTAGE = 20;

/*
CPULimiter:
This class helps to limit the CPU usage/consumption by a thread involving
some kind of repetitive/polling kind of operation in a loop.
The limit can be set by a user through a function of this class.
*/

class CPULimiter
{
    //This integer stores last total system time.
    //total system time is sum of time spent by system 
    //in kernel, user and idle mode
    LARGE_INTEGER m_lastTotalSystemTime;

    //This integer stores last total time spent by this 
    //thread in    kernel space and user space
    LARGE_INTEGER m_lastThreadUsageTime;

    //Variable used to store maximum thread CPU percentage
    //relative to system total time.
    int m_ratio;
public:

    //Constructors
    CPULimiter();
    //Constructor with maximum thread cpu usage percentage
    CPULimiter(int p_ratio);

    //****************Main function.****************** 
    //It evaluates CPU consumption by this thread since 
    //last call to this function, until now. 
    //Internally, it calculates if the thread has exceeded 
    //the maximum CPU usage percentage specified.
    //if yes, it makes the thread sleep for a calculated 
    //time period, to average the total usage to the limit specified.
    //Returns TRUE Successful, else FALSE
    
    BOOL CalculateAndSleep();

    //Inline setter function
    inline void SetRatio(int p_ratio){m_ratio = p_ratio;}
};

And this is the CalculateAndSleep function:

BOOL CPULimiter::CalculateAndSleep()
{
    //Declare variables;
    FILETIME sysidle, kerusage, userusage, threadkern
                   , threaduser, threadcreat, threadexit; 
    LARGE_INTEGER tmpvar, thissystime, thisthreadtime; 

    //Get system kernel, user and idle times
    if(!::GetSystemTimes(&sysidle, &kerusage, &userusage))
        return FALSE;

    //Get Thread user and kernel times
    if(!::GetThreadTimes(GetCurrentThread(), &threadcreat, &threadexit
                            , &threadkern, &threaduser))
        return FALSE;

    //Calculates total system times
    //This is sum of time used by system in kernel, user and idle mode.

    tmpvar.LowPart = sysidle.dwLowDateTime;
    tmpvar.HighPart = sysidle.dwHighDateTime;
    thissystime.QuadPart = tmpvar.QuadPart;

    tmpvar.LowPart = kerusage.dwLowDateTime;
    tmpvar.HighPart = kerusage.dwHighDateTime;
    thissystime.QuadPart = thissystime.QuadPart + tmpvar.QuadPart;

    tmpvar.LowPart = userusage.dwLowDateTime;
    tmpvar.HighPart = userusage.dwHighDateTime;
    thissystime.QuadPart = thissystime.QuadPart + tmpvar.QuadPart;

    //Calculates time spent by this thread in user and kernel mode.

    tmpvar.LowPart = threadkern.dwLowDateTime;
    tmpvar.HighPart = threadkern.dwHighDateTime;
    thisthreadtime.QuadPart = tmpvar.QuadPart;

    tmpvar.LowPart = threaduser.dwLowDateTime;
    tmpvar.HighPart = threaduser.dwHighDateTime;
    thisthreadtime.QuadPart = thisthreadtime.QuadPart + tmpvar.QuadPart;

    //Check if this is first time this function is called
    //if yes, escape rest after copying current system and thread time
    //for further use
    //Also check if the ratio of differences between current and previous times
    //exceeds the specified ratio.
    
    if( thisthreadtime.QuadPart != 0
        && (((thisthreadtime.QuadPart - m_lastThreadUsageTime.QuadPart)*100) 
          - ((thissystime.QuadPart - m_lastTotalSystemTime.QuadPart)*m_ratio)) > 0)
    {
        //Calculate the time interval to sleep for averaging the extra CPU usage 
        //by this thread.

        LARGE_INTEGER timetosleepin100ns;
        timetosleepin100ns.QuadPart = (((thisthreadtime.QuadPart 
                                        - m_lastThreadUsageTime.QuadPart)*100)/m_ratio) 
                       - (thissystime.QuadPart 
                                          - m_lastTotalSystemTime.QuadPart);
        
        //check if time is less than a millisecond, if yes, keep it for next time.
        if((timetosleepin100ns.QuadPart/10000) <= 0)
            return FALSE;
        
        //Time to Sleep :)
        Sleep(timetosleepin100ns.QuadPart/10000);
    }  

    //Copy usage time values for next time calculations.
    m_lastTotalSystemTime.QuadPart = thissystime.QuadPart;
    m_lastThreadUsageTime.QuadPart = thisthreadtime.QuadPart;
    return TRUE;
} 

Here is a sample CPU intensive code using this class:

int _tmain(int argc, _TCHAR* argv[])
{
    std::cout<<"This is a cpu intensive program";
    //define CPULimiter variable
    //and limit cpu usage percentage to 5% here.
    CPULimiter limiter = 5;
    int a,b;
    a = 78;

    //Some CPU intensive code here....
    while(1)
    {    
        b = a;

        //limit cpu usage here.
        limiter.CalculateAndSleep();
    }
    return 0;
}

Note: Here we have confined the usage to 5%. If we hadn't, this would have consumed all the CPU.

CPULimiter limiter = 5; 

History

• 27^th August, 2008: Initial post