Throttling - Managing Application Performance

Introduction

Often, an application is developed that interacts with an external host/application. A situation arises that the application needs to control/throttle the rate at which messages are being sent out or read/received from the external host. It could be a messaging application or a repetitive call to perform a function.

This article would like to address that scenario. It is a spin off from what I have been working on and have been looking to optimize the performance of.

Background

Some scenarios where this could be useful are:

1. Send messages to another host/application at a controlled rate, e.g., 50 messages/second.
2. Read messages from a queue (MQ queue or otherwise) at a controlled rate in a non-blocking processing environment, so that the load on your application is controlled.

Using the code

The abstract (MustInherit) class ProcessGovernor manages the throttling. A new class (clsTestGiov) can be defined by extending (inheriting) ProcessGovernor and implementing a "trigger".

setThrottle is for setting the number of calls/reads to be done in 30 seconds. If setThrottle is uninitialized, it runs without throttling.

//
// Process Governor
//

Public MustInherit Class ProcessGovernor
    '--------------------------------------------------- Constants
    Private Const cTIMESLOT As Short = 30
    '--------------------------------------------------- Params
    Dim objName As String
    '--------------------------------------------------- Throttle Params
    Private timeSlot As Short
    Private shtTotMsg As Short
    Private shtMaxMsg As Short = -1
    '--------------------------------------------------- Throttle Timers
    Dim objTimer As TimerCallback
    Dim CallBackTimer As Timer
    Private evntRead As AutoResetEvent
    '--------------------------------------------------- Process Loop
    Private blnReadyStatus As Boolean

     Public Sub New() 
        timeSlot = GetTimeSlot()
        shtMaxMsg = -1
        objTimer = New TimerCallback(AddressOf RestartTrigger)
        CallBackTimer = New Timer(objTimer, Nothing, _
                            TimeSpan.FromSeconds(2), _
                            TimeSpan.FromSeconds(0))
        Me.evntRead = New AutoResetEvent(False)
    End Sub
    
      Public MustOverride Sub Trigger() 

     Public Sub SetThrottle(ByVal shtCount As Short) 
        Try
            If Not IsNumeric(shtCount) Then
                Exit Sub
            End If
            If (shtCount < 1) Or (shtCount > Short.MaxValue) Then
                shtMaxMsg = -1
            Else
                shtMaxMsg = shtCount
            End If
        Catch ex As Exception
            shtMaxMsg = -1
        End Try
    End Sub

     Public Function GetThrottle() As Short  
        Return shtMaxMsg
    End Function

    Public Sub SetName(ByVal strName As String)
        objName = strName
    End Sub

    Public Function GetName() As String
        Return objName
    End Function

    Public Sub StartGovr()
        blnReadyStatus = True
        While blnReadyStatus
            If shtMaxMsg <> -1 Then
                If timeSlot <> GetTimeSlot() Then
                    timeSlot = GetTimeSlot()
                    shtTotMsg = 1
                Else
                    shtTotMsg += 1
                End If
                If shtTotMsg > shtMaxMsg Then
                    CallBackTimer.Change(GetTimeSlotExpiry() * 1000, 0)
                    evntRead.WaitOne()
                Else
                    Trigger()
                End If
            Else
                Trigger()
            End If
        End While
    End Sub

    Public Sub StopGovr()
        If blnReadyStatus Then
            blnReadyStatus = False
        End If
    End Sub

    Public Sub ShutDownGovr()
        blnReadyStatus = False
        CallBackTimer.Dispose()
        objTimer = Nothing
        evntRead = Nothing
    End Sub

    Private Sub RestartTrigger(ByVal inpObjState As Object)
        CallBackTimer.Change(System.Threading.Timeout.Infinite, 0)
        evntRead.Set()
    End Sub

    Private Function GetTimeSlot() As Short
        Return (Now.Hour * 60 + Now.Minute) * 10 + Int(Now.Second / cTIMESLOT)
    End Function

    Private Function GetTimeSlotExpiry() As Short
        Return (cTIMESLOT - (Now.Second Mod cTIMESLOT))
    End Function

    Protected Overrides Sub Finalize()
        blnReadyStatus = False
        If Not objTimer Is Nothing Then
            objTimer = Nothing
        End If
        If Not CallBackTimer Is Nothing Then
            CallBackTimer.Change(Threading.Timeout.Infinite, 0)
            CallBackTimer.Dispose()
        End If
        If Not evntRead Is Nothing Then
            evntRead.Reset()
            evntRead = Nothing
        End If
    End Sub

End Class


//
// Class to  implement Governor/Throttle 
// 
Public Class clsTestGov  
    Inherits ProcessGovernor
    '--------------------------------------------------- Throttle Params
    Dim objProcessor As clsProcessor
    Public Sub New(ByVal inProcessor As HostProcessor)
        MyBase.New()
        objProcessor = inProcessor
    End Sub
    Public Overrides Sub Trigger()
        objProcessor.QueMsg(KapitiHost.GetMessage)
        Thread.Sleep(100)
    End Sub
End Class

//
// Start Processing using the Governor. 
// Note that the Governor is started in a separate thread 
//
Public Class clsProcessor
  
    '-------------------------------------------------- Governor
    Private objGovernor As ProcessGovernor
    Dim t As Thread
     Public Sub New()
        objGovernor = New clsTestGov (Me)
        //
        // Set throttle to 50 messages in 30 seconds
        //
        objGovernor.SetThrottle(50)
    End Sub
    Public Sub StartListeningGov()
        Try
            t = New Threading.Thread(New _
                Threading.ThreadStart(AddressOf objGovernor.StartGovr))
            t.Start()
        Catch ex As Exception
            MsgBox(ex.Message)
        End Try
    End Sub
    Public Sub StopListeningGov()
        objGovernor.ShutDownGovr()
        Try
            t.Abort()
        Catch ex As Exception
            MsgBox(ex.Message)
        End Try
        t.Join()
        t = Nothing
    End Sub
End Class

To implement, associate the clsTestGov in a parent class (clsProcessor) and call the method "StartGovr" on a new thread. Set the throttle (the number of calls) to 30 seconds.

Points of interest

Callback timers and WaitHandles are extremely useful, and a good understanding of them opens up your choices while designing a system.