Click here to Skip to main content
6,634,665 members and growing! (20,631 online)
Email Password   helpLost your password?
General Reading » Hardware & System » General     Intermediate

Intercepting WinAPI calls

By Andriy Oriekhov

An article about intercepting WinAPI calls.
VC6, VC7Win2K, WinXP, Win2003, Dev
Posted:6 Mar 2006
Updated:31 May 2006
Views:59,143
Bookmarked:69 times
Announcements
Loading...
 
Search    
Advanced Search
Add to IE Search
printPrint   add Share
      Discuss Discuss   Broken Article?Report  
27 votes for this article.
Popularity: 6.68 Rating: 4.67 out of 5

1
1 vote, 3.7%
2

3
4 votes, 14.8%
4
22 votes, 81.5%
5

Introduction

API calls interception is the task that allows to get access to some parts of other's programs. Lots of programmers spend time on developing and describing various methods which allow that access. Such methods are used in many anti-viruses and anti-spyware. Besides, sometimes, intercepting can help you to find errors in your application. However, it is not a secret that some viruses use it too. I spent much time to find and understand the technique of interception. I would like to describe here the results of my research.

Method description

First of all, you need to read the following article to understand the basics of the interception mechanism: HookSys (written by Ivo Ivanov). It was very helpful for me, and I used the sample code from it. However, it does not solve all my problems because Ivo's samples sometimes miss very important API calls. It happens when the application starts up too fast and the intercepting service has no time to inject the DLL. After some research, I found the actual problem, and it was related to using the kernel mode function, SetCreateProcessNotificationRoutine. This function is used to receive notification events about any new process creation. Such a notification is often fired when the process has already been started. Therefore, I needed to find a way to improve Ivo's code.

As far as I know, the execution of all Windows processes consists of the following steps:

  • initial process loading;
  • creating the main thread for the process in the suspended state;
  • mapping of the NT.DLL into the address space of the process;
  • mapping all needed DLLs, and calling their DllMain with the DLL_PROCESS_ATTACH reason;
  • resuming the main process' thread.

The step right before the main thread resuming looks like the most comfortable for injection because the process is in suspended state and none of its instructions have been executed yet.

Most of the work on the process creation is done in the kernel mode, so to change this algorithm, you need to intercept the kernel mode functions NtCreateProcess() and NtCreateThread(). The CONTEXT structure, the pointer to which is passed to the function NtCreateThread(), contains a member called EAX. I found that it equals to the process' start address in user mode, so if you can change it, then you can get the control right after process creation and before starting. To solve this task, I wrote a kernel mode driver. It starts while the system starts up.

There are some initialization steps:

  1. starting;
  2. receiving configuration from the user mode;
  3. intercepting kernel mode functions such as: NtCreateProcess(), NtCreateThread(), NtTerminateProcess(), NewNtCreateProcessEx() - for Windows 2003 Server.

A handler to the NtCreateThread() function contains code that will do most of the interesting jobs. Here is a brief description of its algorithm:

  1. allow access to the creating process by calling ObReferenceObjectByHandle();
  2. remember the main thread start address (ThreadContext->EAX);
  3. "jump" to the context of the creating process by calling KeAttachProcess();
  4. allocate memory for my code by calling ZwAllocateVirtualMemory(), similar to the well known technique for CreateRemoteThread() in user mode;
  5. copy the small code to the allocated memory that will load my DLL. This code looks like:
    push pszDllName
    mov  ebx, LoadLibraryAddr
    call [ebx]
    mov  eax, Win32StartAddr
    push eax
    ret
    pszDllName: db 'example.dll';
  6. "jump" to the initial process;
  7. change the thread start address (ThreadContext->EAX) so it will point to the allocated memory.

That is all. You can download and compile the complete source code for this article. Note: the sample is fully functional and quite enough for basic understanding, but for real usage, it might be rewritten.

Compiling the code

You need the NTDDK to be installed on your computer. I'm using MSVS 6.0 for compiling NtProcDrv, and MSVS 7.1 for the rest of the projects.

History

  • 2006-03-06 - Submitted.
  • 2006-05-31 - Source codes and binaries are updated.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

About the Author

Andriy Oriekhov


Member

Occupation: Chief Technology Officer
Location: Ukraine Ukraine

Other popular Hardware & System articles:

Article Top
You must Sign In to use this message board.
FAQ FAQ 
 
Noise Tolerance  Layout  Per page   
 Msgs 1 to 25 of 42 (Total in Forum: 42) (Refresh)FirstPrevNext
QuestionProblem in Release Build Pinmembernoumantariq0:49 29 Oct '09  
GeneralCorrection in assembly code [modified] PinmemberMattsUserName16:04 27 Jul '09  
GeneralVista compatibility? PinmemberAmit Vilas Shinde2:36 7 Mar '08  
GeneralHi, tezm. I ran into the same question as yours. Pinmembermybayern197423:02 2 Aug '07  
GeneralRe: Hi, tezm. I ran into the same question as yours. Pinmembertezm7:55 4 Aug '07  
GeneralRe: Hi, tezm. I ran into the same question as yours. Pinmembermjmim13:34 10 Oct '08  
QuestionError installing service, how to solve it? Pinmembertezm9:29 27 Jul '07  
AnswerRe: Error installing service, how to solve it? Pinmembertezm2:28 28 Jul '07  
GeneralI want to support vista O.S., please help me PinmemberHsiao, Tsu Tair17:46 25 Jul '07  
GeneralInjection works fine, but impossible to subclass using this method? [modified] Pinmemberehaerim12:05 5 Jan '07  
I've modified the dllhookapi.cpp to see
1) if I can subclass Notepad.exe successfully within DllMain.
2) if I can subclass multiple instances of Notepad.exe successfully.

The code works this way:
1) Call EnumProcesses to find out all the running processes.
2) Call Subclass with the process id array obtained above and the name of notepad for subclassing.
3) For each process id, Subclass will call EnumProcessModules and GetModuleFileNameEx to get the full module name and repeat this until we find notepad.
4) Now if we have found notepad, call EnumThreadWindows with current thread id to find out the main window to subclass.

Here the problem rises. EnumThreadWindows returns successfully, but EnumThreadWindowsProc NEVER get called. I couldn't understand why. Now I guess it is because somehow the current thread did not create any windows yet. So EnumThreadWindows return TRUE but no EnumThreadWindowsProc call!

But then, how can I subclass notepad right after the creation?

haerim



// dllhookapi.cpp : Defines the entry point for the DLL application.
//
///////////////////////////////////////////////////////
// Andriy Oriekhov. 2006. Toleron Sofware.
// www.toleron.com
///////////////////////////////////////////////////////

#include "stdafx.h"
#include "windows.h"

#include



HWND g_hwnd = NULL;
BOOL CALLBACK EnumThreadWindowsProc(HWND hwnd, LPARAM lParam);
// New & old window procedure of the subclassed window
WNDPROC OldProc = NULL;
LRESULT CALLBACK NewProc(HWND, UINT, WPARAM, LPARAM);


void Subclass(DWORD* paProcess, DWORD nCount, LPCTSTR pName);


BOOL WINAPI DllMain(HANDLE hModule, DWORD ul_reason_for_call, LPVOID lpReserved)
{
switch (ul_reason_for_call) {
case DLL_PROCESS_ATTACH:
{
WriteLogDebug("Before starting process.");

// Get the list of process identifiers.
DWORD aProcesses[1024], cbNeeded, cProcesses;
if (EnumProcesses(aProcesses, sizeof(aProcesses), &cbNeeded) == 0)
{
DWORD dwErr = GetLastError();
WriteLogDebug("DllMain: dwErr=%d", dwErr);
return TRUE;
}

// Find the matching process and subclass it.
cProcesses = cbNeeded / sizeof(DWORD);
Subclass(aProcesses, cProcesses, "C:\\Windows\\System32\\notepad.exe");

break;
}
case DLL_PROCESS_DETACH:
{
WriteLogDebug("Before ending process.");
if (::SetWindowLongPtr(g_hwnd, GWL_WNDPROC, (long)OldProc) == 0)
{
WriteLogDebug("Failed to unsubclass back from NewProc(0x%X) to OldProc(0x%X)", GetWindowLongPtr(g_hwnd, GWL_WNDPROC), (long)OldProc);
}

break;
}}

return TRUE;
}

void Subclass(DWORD* paProcess, DWORD nCount, LPCTSTR pName)
{
WriteLogDebug("Subclass: nCount=%d, pName=%s", nCount, pName);
char szProcessName[MAX_PATH] = "";

unsigned int i;
for (i = 0; i < nCount; i++)
{
// Get a handle to the process.
DWORD dwPID = paProcess[i];
HANDLE hProcess = OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_VM_READ, FALSE, dwPID);

if (NULL == hProcess)
{
WriteLogDebug("Subclass: dwPID=0x%X(%d), OpenProcess failed!", dwPID, dwPID);
CloseHandle(hProcess);
continue;
}

// Get modules for this process
DWORD cbNeeded = 0;
HMODULE hMod;
if (EnumProcessModules(hProcess, &hMod, sizeof(hMod), &cbNeeded) == 0) //
{
WriteLogDebug("Subclass: hProcess=0x%X, EnumProcessModules failed!", hProcess);
continue;
}
DWORD nModCount = cbNeeded / sizeof(HMODULE);
WriteLogDebug("Subclass: nModCount=%d", nModCount);

// Get the process name.
GetModuleFileNameEx(hProcess, hMod, szProcessName, sizeof(szProcessName));
WriteLogDebug("Subclass: dwPID=0x%X(%u), cbNeeded=%d, %s\n", dwPID, dwPID, cbNeeded, szProcessName);
if (strcmpi(szProcessName, pName) != 0)
{
CloseHandle(hProcess);
continue;
}
/*
// Go through with all the windows for the current thread
// => There is no windows for dwPID yet. So, EnumWindowsProc will NEVER be called for dwPID.
EnumWindows(EnumWindowsProc, (LPARAM)dwPID);
*/

DWORD dwTID = GetCurrentThreadId();
WriteLogDebug("Subclass: dwTID=0x%X", dwTID);
// Now, let's subclass the main window. => This seems not working. See comments below.
// The below EnumThreadWindows seems to NEVER call EnumThreadWindowsProc because there is no windows for the current thread created yet.
// So, EnumThreadWindows call succeeds returning TRUE but no EnumThreadWindowsProc gets called at all.
// Then how can I subclass the main window of this thread? I don't know the answer yet.
if (EnumThreadWindows(dwTID, EnumThreadWindowsProc, (LPARAM)dwPID) == 0)
{
DWORD dwErr = GetLastError();
WriteLogDebug("Subclass: dwTID=0x%X, dwErr=%d, EnumThreadWindows failed!", dwTID, dwErr);
CloseHandle(hProcess);
continue;
}

CloseHandle(hProcess);
}
}

BOOL CALLBACK EnumThreadWindowsProc(HWND hwnd, LPARAM lParam)
{
WriteLogDebug("EnumThreadWindowsProc(0x%X, %d)", hwnd, lParam);

// Let's find hwnd's process is the same as DllMain's calling process
DWORD dwPID_hwnd = 0, dwTID_hwnd = 0;
dwTID_hwnd = GetWindowThreadProcessId(hwnd, &dwPID_hwnd);

TCHAR wmfn[MAX_PATH + 1] = {0};
UINT nLen = GetWindowModuleFileName(hwnd, wmfn, MAX_PATH); // Huh, this does not work for other processes, but only for the calling process!!!
WriteLogDebug("EnumThreadWindowsProc: hwnd=0x%X, dwPID_hwnd =0x%X(%d), dwTID_hwnd=0x%X(%d)\n"
" nLen=%d, wmfn=%s",
hwnd, dwPID_hwnd, dwPID_hwnd, dwTID_hwnd, dwTID_hwnd,
nLen, wmfn);
return TRUE;
// // Finally, let's subclass hwnd
// WNDPROC OldProc = (WNDPROC)::SetWindowLongPtr(hwnd, GWL_WNDPROC, (long)NewProc);
// if (OldProc == 0)
// WriteLogDebug("EnumThreadWindowsProc: Failed to subclass hwnd(0x%X)", hwnd);
// else
// WriteLogDebug("EnumThreadWindowsProc: Succeeded in subclassing hwnd(0x%X)'s OldProc(0x%X) to NewProc(0x%X)",
// hwnd, (long)OldProc, (long)NewProc);
//
// return FALSE;
}

//-------------------------------------------------------------
// NewProc
// Notice: - new window procedure for the START button;
// - it just swaps the left & right muse clicks;
//
LRESULT CALLBACK NewProc(
HWND hwnd, // handle to window
UINT uMsg, // message identifier
WPARAM wParam, // first message parameter
LPARAM lParam // second message parameter
)
{
// WriteLogDebug("wParam=%d, lParam)=%d", wParam, lParam);
switch (uMsg)
{
//WM_COMMAND nNotifyCode:0 (Menu) wID:34795
case WM_COMMAND:
WriteLogDebug("wParam=%d, lParam, HIWORD(wParam)(nNotifyCode)=%d, LOWORD(lParam)(id)=%d=%d", wParam, lParam, HIWORD(wParam), LOWORD(lParam));
if (HIWORD(wParam) == 0)
{
// WriteLogDebug("2,HIWORD(wParam)(nNotifyCode)=%d, LOWORD(lParam)(id)=%d", HIWORD(wParam), LOWORD(lParam));
switch(LOWORD(wParam))
{
case 34795:
// display main dialog
::MessageBox(NULL,"NULL","hStart",MB_OK);
return S_OK;
case 34796:
return S_OK;
case 34797:
return S_OK;
default:
break;
}
}
}

return CallWindowProc(OldProc,hwnd,uMsg,wParam,lParam);
}


// Usage
//WriteLog("int value=%d \n str value=%s", 19, "test");
#include
void inline WriteLog(char *ch, ...)
{
char buf[1024];
va_list arg_list;

va_start( arg_list, ch );
wvsprintf( buf, ch, arg_list );
va_end( arg_list );

FILE *file;
file = fopen("dllhookapi.log", "a+");
fprintf(file, "%s", buf);
fclose(file);
}

void inline WriteLogDebug(char *ch, ...)
{
char buf[1024];
va_list arg_list;

va_start( arg_list, ch );
wvsprintf( buf, ch, arg_list );
va_end( arg_list );

OutputDebugString(buf);
}






-- modified at 17:11 Friday 5th January, 2007
Sign In·View Thread·PermaLink
GeneralHow to inject dll into specific processes, not all the processes? [modified] Pinmemberehaerim22:28 30 Dec '06  
GeneralRe: How to inject dll into specific processes, not all the processes? Pinmemberehaerim11:35 5 Jan '07  
GeneralNewNtTerminateProcess called with NULL as the value of ProcessHandle. Why? Pinmemberehaerim21:59 30 Dec '06  
GeneralSecond MessageBox does not show up when process termination but only sound. Pinmemberehaerim15:56 2 Dec '06  
GeneralRe: Second MessageBox does not show up when process termination but only sound. Pinmembertezm2:25 28 Jul '07  
GeneralVC++ 6.0 Version available? Pinmemberehaerim11:52 2 Dec '06  
GeneralHow to hook multiple instances of Notepad? [modified] Pinmemberehaerim16:10 30 Nov '06  
GeneralWhere can I get info about SetCreateProcessNotificationRoutine? [modified] Pinmemberehaerim14:42 30 Nov '06  
Generalstarting programs by clicking on document icons Pinmemberrichardmoss6:43 15 Nov '06  
GeneralRe: starting programs by clicking on document icons PinmemberAndriy Oriekhov1:25 23 Nov '06  
GeneralHow to get the executable name from the created process?? Pinmembergirm2:41 13 Sep '06  
GeneralRe: How to get the executable name from the created process?? PinmemberAndriy Oriekhov3:53 14 Sep '06  
GeneralRe: How to get the executable name from the created process?? Pinmembergirm21:40 24 Oct '06  
GeneralRe: How to get the executable name from the created process?? PinmemberAndriy Oriekhov12:32 25 Oct '06  
GeneralRe: How to get the executable name from the created process?? Pinmemberehaerim11:32 5 Jan '07  

General General    News News    Question Question    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

PermaLink | Privacy | Terms of Use
Last Updated: 31 May 2006
Editor: Smitha Vijayan
Copyright 2006 by Andriy Oriekhov
Everything else Copyright © CodeProject, 1999-2009
Web21 | Advertise on the Code Project