Click here to Skip to main content
15,867,686 members
Articles / Programming Languages / C++

A (working) implementation of API hooking (Part I)

Rate me:
Please Sign up or sign in to vote.
3.27/5 (10 votes)
24 Mar 2005CPOL5 min read 117.1K   3.4K   72   25
This article provides a very simple but efficient base code for hooking API functions in any application.

Sample Image - APIHookingRevisited.jpg

Introduction

There are numerous codes in the WWW to explain how to hook API functions inside a remote process. So, why creating another one?

This article is the first (I) part of a tool I'm working on. These tools won't be too complex, so there is no need for a huge class hierarchy like in almost all other codes, nor a hard coded assembly code. This tool is just a technical preview of what is possible to achieve, with QUICK and DIRTY "put your functions here" blocks.

Before you read

Okay, to be clear, when you'll open the project and have a look to the code, you'll see it is not very well polished. It doesn't compile without warning, there is only few comments (compared to production code). However, it compiles and run cleanly on WinXP and 2003 platforms, (and should also run under Win2K, but I didn't check that).

Background

I used to develop a software using DirectShow. Sadly, I run into problem with a simple synchronization object which deadlocked the application. I still think there are two things a developer might fear about.

  1. The first one is "memory leaks". Memory leaks are most of the time due to an oversight. They are very well covered because they are easily (automatically) corrected.
  2. The other one is "error handling". Error handling seems very simple at first, but most of the time, there is always the "BAD CASE" which causes the "BAD BUG".

Badly handling errors in a software leads to crash or illogic state. To compare to physic state, it is easy to predict and create a system in its stable state. However, reaching/leaving this stable state is hard to predict, because error / thread scheduling order / memory usage / dark phenomena might change parameters. Recovering from an unstable state is very hard.

I can't give a global solution to this (if anyone could, please mail me). So let's see the usual solutions:

  • Error return: Never expect a function will always work. There are numerous articles about these issues.
  • Thread: This is the subject of the whole project.
  • Memory usage: Monitoring the memory allocated by overriding the malloc/free/new/delete functions/operators most of time is sufficient.
  • IO states: You'll need to establish a complete graph of states for handling all cases

Threads are one of the most "strange" thing. While it is possible to simulate almost all memory conditions, IO states and error returns, it is not possible to simulate the execution order of threads.

(uh, that's long!)

So to sum up, this project only focuses on one problem: data synchronisation in multithreaded environment. This article explains a technique to monitor calls to system functions. The aim of the global project is to build a deadlock detector. A deadlock occurs when, while protecting the access to a block of data (A), a thread is trying to access another protected block of data (B). This other block (B) is protected by another thread which tries to access the first protected block (A).

Then, to achieve this goal, let's split the project in small steps:

  • Part 1. Hooking system functions.
  • Part 2. Intercepting all calls to synchronization functions.
  • Part 3. Building the logic for deadlock detection.
  • Part 4. [Cosmetic] Using the map file + stack address to fetch the source code[/Cosmetic].

Part 1. Hooking system functions

I highly recommend you to read "Process-wide API spying - an ultimate hack" article by Anton Bassov and "API Hooking system" by Ivo Ivanov.

I won't dig into an n-th explanation about how to inject your code into a process. I will however explain what I chose, and how I make it works for me. The idea is to inject a DLL in a remote process so that the DLL will perform the function hijacking. The DLL is called in this project ThreadSpy.DLL. When the DLL is created, the DllMain function is called with ProcessAttach's reason. In this case, it parse the HookStruct array (defined in ThreadSpy.cpp), and replaces a given Win32 API function by a DLL supplied one.

The hook functions are defined in Hooked.cpp source file. Then, the import table of each already loaded module (like Kernel32.DLL, USER32.DLL, GDI32.DLL, etc...) is updated to replace the original function address with any supplied function.

Then a server could inject this DLL in a running process or spawn its own. The server is inside the ThreadDLD project, and is simply a WTL application which will receive message from the hooked process and inject the DLL in the hooked process. I've chosen to create the monitored process with the CreateProcess function with CREATE_SUSPENDED flag. This way, I can easily inject my DLL before any other DLL are loaded (in fact after the vital one like NTDLL and Kernel32). Then the server resume the thread, and is ready to collect the message from the DLL.

In order to intercept any newly loaded DLL in the hooked process, it is required to intercept the LoadLibrary and GetProcAddress functions. That's why those functions are always hooked.

I won't discuss about how to send a message to the server in this article, that's why I removed the code for this in the demo and source. With this demo, you could easily implement your own API hooking by changing the functions in ThreadSpy.{h,cpp} and in Hooked.cpp. This way you can easily monitor who is trying to create a file (and reject the creation if the credentials are not high enough) by adding a MyCreateFile function.

In this example, I've chosen to hook TextOutA and TextOutW so that it is visually obvious.

Credits

As I said earlier, please read:

Future...

In the second part of the project, I will try to explain how to intercept all the required functions to monitor thread creation/destructions and synchronisation objects and communicate those information to the server. This article is more a prolog to the real article. The second part is here: Thread deadlock detection (part II)

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
France France
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
GeneralExelent code!. Unble to hook user32.dll >GetMessageW Pin
alphadan14-Nov-10 16:10
alphadan14-Nov-10 16:10 
GeneralWell i have some issues with this hook, seems like it dosent hook virtualquery and virtualqueryex i did exactly like u did but they wont just getting hooked Pin
nah13374-Sep-09 13:24
nah13374-Sep-09 13:24 
GeneralInteresting Link collection Pin
Elmue29-Sep-08 17:05
Elmue29-Sep-08 17:05 
GeneralUnhookFunctions Pin
ste82716-Jan-07 6:46
ste82716-Jan-07 6:46 
GeneralRe: UnhookFunctions (fix?) Pin
pablodg18-May-07 22:58
pablodg18-May-07 22:58 
GeneralHooking Library during LoadLibrary Pin
mridey26-Apr-06 21:52
mridey26-Apr-06 21:52 
GeneralHandling of ordinals Pin
mridey26-Apr-06 21:49
mridey26-Apr-06 21:49 
QuestionHook API WriteProcessMemory or OpenProcess ? Pin
Vitoto2-Jan-06 8:29
Vitoto2-Jan-06 8:29 
AnswerRe: Hook API WriteProcessMemory or OpenProcess ? Pin
xryl66923-Feb-06 5:13
xryl66923-Feb-06 5:13 
GeneralRe: Hook API WriteProcessMemory or OpenProcess ? Pin
Vitoto23-Feb-06 5:24
Vitoto23-Feb-06 5:24 
thank you man, we think how using that.

You are expert in that.

You have some idea "How detect when clients using programs XP SPEED" ?
This programas accelerate the timer and run more fast inside game.

Any idea where begin my search ?

GeneralRe: Hook API WriteProcessMemory or OpenProcess ? Pin
xryl66927-Feb-06 0:31
xryl66927-Feb-06 0:31 
GeneralStackWalk64 issue Pin
rajandpayal27-Jul-05 4:42
rajandpayal27-Jul-05 4:42 
GeneralRe: StackWalk64 issue Pin
xryl6691-Nov-05 4:20
xryl6691-Nov-05 4:20 
GeneralUnhooking Pin
Kharfax19-Jul-05 10:14
Kharfax19-Jul-05 10:14 
GeneralRe: Unhooking Pin
xryl6691-Nov-05 4:15
xryl6691-Nov-05 4:15 
GeneralUsing Notepad.exe with ThreadSpy.exe Pin
sun_hcl16-May-05 2:02
sun_hcl16-May-05 2:02 
GeneralRe: Using Notepad.exe with ThreadSpy.exe Pin
xryl66918-Jul-05 1:34
xryl66918-Jul-05 1:34 
GeneralFindFunction crash Pin
ravenspoint30-Mar-05 7:42
ravenspoint30-Mar-05 7:42 
GeneralRe: FindFunction crash Pin
xryl6691-Apr-05 23:19
xryl6691-Apr-05 23:19 
QuestionSome Questions ??? Pin
vishalmore3-Mar-05 17:28
vishalmore3-Mar-05 17:28 
AnswerRe: Some Questions ??? Pin
X-Ryl66924-Mar-05 23:40
sussX-Ryl66924-Mar-05 23:40 
GeneralDoes not compile with w2k Pin
Member 89978418-Feb-05 3:27
Member 89978418-Feb-05 3:27 
GeneralRe: Does not compile with w2k Pin
vishalmore1-Mar-05 18:17
vishalmore1-Mar-05 18:17 
Generalcannot open file "Dbghelp.lib" Pin
Jetli Jerry31-Jan-05 20:35
Jetli Jerry31-Jan-05 20:35 
GeneralRe: cannot open file "Dbghelp.lib" Pin
xryl6691-Feb-05 6:25
xryl6691-Feb-05 6:25 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.