QueueUserAPCEx: Extending Win32 User-Mode Asynchronous Procedure Calls (APCs)

• Download source files - 10 Kb
• Download demo project - 23 Kb

Introduction

In this work, we implement an extended version of the QueueUserAPC function, called QueueUserAPCEx, that provides truly asynchronous user-mode procedure calls without requiring the target thread to call any special function in order to enter an alertable state.

Background

For asynchronous notification, Windows supports Asynchronous Procedure Calls (APCs). An APC is a kernel-defined control object that represents a procedure that is called asynchronously. APCs have the following features [1]:

• An APC always runs in a specific thread context.
• An APC runs at OS predetermined times.
• They can cause preemption of the currently running thread.
• APC routines can be preempted.

There are three different types of APCs in the kernel [2]: User mode, Normal Kernel mode and Special Kernel mode APCs. Kernel mode APCS are executable by default. On the other hand, User mode APCS are, by default, disabled; that is, they are queued to the user-mode thread, but are not executed, except at well-defined point in the program. Specifically, they can be executed only when an application has either called a wait service and has enabled alerts to occur or called the test-alert service.

The Win32 API provides the QueueUserAPC function, which allows an application to queue an APC object to a thread. The queuing of an APC is a request for the thread to call the APC function. When a user-mode APC is queued, the thread is not directed to call the APC function unless it is in an alertable state. Unfortunately, a thread enters an alertable state only by using one of the following Win32 API functions: SleepEx, SignalObjectAndWait, WaitForSingleObjectEx, WaitForMultipleObjectEx, or MsgWaitForMultipleObjectsEx.

Using the code

A programmer can use the two files (QueueUserApcEx.c and QueueUserApcEx.h) in order to use the QueueUserAPCEx function, which has the same prototype with the ordinary Win32 QueueUserAPC function:

DWORD QueueUserAPCEx(PAPCFUNC pfnApc, HANDLE hThread, DWORD dwData)

Of course, it is necessary to install and load (start) the ALERTDRV.SYS driver first. This can be performed very easily by following the directions of the Readme1st.txt file that is included in the demo project. Another file (Build_Install.txt in the source code distribution, provides directions for building and installing the kernel-mode driver and running successfully the application.

/* testapp.c : demo of QueueUserAPCEx */
#define _WIN32_WINNT 0x0500
#include <windows.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
 
#include "QueueUserApcEx.h"

INT ExitFlag = 0;
 
DWORD WINAPI ThreadRoutine(LPVOID lpParam)
{
  DOUBLE t1, t2;

  printf("[Thread %ld] Starting\n", GetCurrentThreadId());
  t1 = GetTickCount();
  t2 = t1 + 0.5*1000;    /* Every 0.5 seconds print something */

  while (!ExitFlag) {
    if (t1 > t2) {
      printf("[Thread %ld] Running...\n", GetCurrentThreadId());
      t2 = t1 + 0.5*1000;
    }
    t1 = GetTickCount();
  }
  printf("[Thread %ld] Exiting\n", GetCurrentThreadId());
  return 0;
}
 
DWORD WINAPI APCRoutine(DWORD APCParam)
{
  printf("[Thread %ld] Inside APC routine with argument (%ld)\n", 
  GetCurrentThreadId(), APCParam);
  return 0;
}
 
int main(void)
{
  DWORD   APCData;
  HANDLE  hThread;
  ULONG   id;
  INT     i, res;

  hThread= CreateThread(NULL, 0, ThreadRoutine, NULL, 0, &id);
  printf("[Thread %ld] Starting\n", GetCurrentThreadId());
    
    
  /* Test: send an APC to myself */ 
  printf("[Thread %ld] Sending an APC to myself\n", GetCurrentThreadId());
  APCData = 33;
  res= QueueUserAPCEx((PAPCFUNC) APCRoutine, GetCurrentThread(), APCData); 
    
    
  /* Sleep for a while; then send 5 APCs to hThread, one every second */
  Sleep(1000);
  for (i = 0; i < 5; i++) {
    printf("[Thread %ld] Sending an APC to the other thread\n", 
      GetCurrentThreadId());
    APCData = i;
    res= QueueUserAPCEx((PAPCFUNC) APCRoutine, hThread, APCData); 
    Sleep(1000);
  }      
    
  ExitFlag = 1;
  WaitForSingleObject(hThread, INFINITE);
    
  printf("[Thread %ld] Exiting\n", GetCurrentThreadId());
  return 0;
}

If you compile and run the above program (testapp.dsp), provided you have started the driver, your output will be like this:

[Thread 1936] Starting
[Thread 2116] Starting
[Thread 1936] Sending an APC to myself
[Thread 1936] Inside APC routine with argument (33)
[Thread 2116] Running...
[Thread 1936] Sending an APC to the other thread
[Thread 2116] Inside APC routine with argument (0)
[Thread 2116] Running...
[Thread 2116] Running...
[Thread 1936] Sending an APC to the other thread
[Thread 2116] Inside APC routine with argument (1)
[Thread 2116] Running...
[Thread 2116] Running...
[Thread 1936] Sending an APC to the other thread
[Thread 2116] Inside APC routine with argument (2)
[Thread 2116] Running...
[Thread 2116] Running...
[Thread 1936] Sending an APC to the other thread
[Thread 2116] Inside APC routine with argument (3)
[Thread 2116] Running...
[Thread 2116] Running...
[Thread 1936] Sending an APC to the other thread
[Thread 2116] Inside APC routine with argument (4)
[Thread 2116] Running...
[Thread 2116] Running...
[Thread 2116] Exiting
[Thread 1936] Exiting
Press any key to continue

Implementation

QueueUserAPCEx differs from QueueUserAPC in that sets the target thread in alertable state. Specifically, the QueueUserAPCEx function calls the ordinary Win32 API QueueUserAPC function and next, through a kernel mode device driver (alertdrv.sys) manages to set the target thread in alertable state. When a thread sends an APC to another, it also sends transparently (through DeviceIoControl) the handle of the target thread to the underlying device driver. The driver’s interrupt service routine receives the information, finds the corresponding ETHREAD structure for that handle (ObReferenceObjectByHandle) and sets the thread in alertable stable. This is possible by setting equal to 1 the memory address that corresponds to 0x4a bytes offset [3] from the base address of the ETHREAD data structure [4].

When a user-mode APC is passed to a thread in this way, its routine will be called next time the thread runs in user-mode. If the thread already runs in user-mode, the routine won't be invoked immediately, however; it'll be called next time the thread _returns_ to user-mode from some kernel-mode service. Usually that happens almost immediately since an active thread is calling kernel-mode services all the time; in the worst case it'll happen after the next clock tick, when clock tick procedure returns to user-mode. The user-mode APC will not, however, interrupt any kernel-mode activity; i.e. if the thread is waiting on an object, it won't be woken up; when it wakes up by itself, however, it will receive the pending APC immediately [3].

For a more advanced implementation of the above functionality, based on Kernel-Mode APCs, and how this is exploited for providing user-defined Unix signals on Windows platforms, see [5].

History

• May 23, 2003 - Initial release.
• Oct 02, 2003 - First update.

References

1. User mode APCs
2. D. A. Solomon. Inside Windows NT, Second Edition, Microsoft Press, 1998.
3. Microsoft Corporation, Microsoft Developer Network Library
4. http://www.cmkrnl.com/arc-userapc.html
5. Panagiotis E. Hadjidoukas, A Device Driver for W2K Signals, Windows Developer Magazine, Volume 12, Number 8, August 2001.