Memory Hooks is a tool for easy detection of memory leaks in any Windows application. Main features:
- No modifications in source code. Actually the code is not required.
- Works for any Windows application written in any language.
- Attaches to any running process.
- Especially effective for applications working in cyclic patterns.
- Aggregation of memory leaks by call stack.
- Inserting breakpoints for easy debugging.
Memory Hooks can be used as a library, or as stand-alone tool using the provided VB wrapper.
Memory Leaks - Different Approach
What is "memory leak"? Usually we mean "a block of memory that was allocated by the program, and was not released". To be more precise we should add: "and was not released before the program ended".
I think this definition is not appropriate for many applications. First let's remember that when a program ends, all the memory that was allocated by it is automatically released by the OS. So it will not make much difference if the memory was released by the process just before the termination. The more interesting parameter, especially for long-running programs, is the change in memory consumption in time. As a simple example, we can look at programs that work in "cyclic" patterns. Here cycle refers to a unit of work, after which the process should return to the state in which it was before the cycle started. For instance: a text editor which opens and works with a document, and then closes it and a service which processes a single request. In all those cases we can talk about "memory leak per cycle". In case such memory leaks exist, even if relatively small, it can cause serious performance problems over time.
Conclusions that we may get with this approach:
- It is more important to avoid cyclic memory leaks, than one-time leaks - O(n) VS O(1).
- Such memory leaks may exist even in programs written in languages featuring automatic garbage collection. For example, in VB you can continuously create new objects, and add them to a Collection, so you will always have a live reference to an object.
Memory leaks detection process
Prior to selecting the right tool for hunting cyclic memory leaks, let's think what additional important characteristics we would like it to have. We'll take a text editor "my_edit.exe" application as an example. I would like to perform the following steps:
- run "my_edit.exe".
- open existing document, add new line, save changes and close the document.
- start monitoring memory allocations.
- repeat step 2.
- stop monitoring memory allocations.
- repeat step 2.
- report: all memory allocations done in step 6, which was not released in steps 6-8.
Here I would like to explain some of the steps in more details:
- Step 2: before starting memory monitoring, I want to make sure the application is fully initialized. It's possible that there will be some one-time allocations while opening the first document.
- Step 6: here I let the application a chance to do full cleanup of memory allocated in step 4. There may be, for example, some objects remaining in memory after step 4, which will be released only when the next document is processed.
Required features - a wish list
Given scenario implies the following features of the memory monitoring tool:
- it should be able to begin the monitoring at some point in time.
- it should be able to stop registering new allocations, but still watch memory being released.
I can extend this "wish list" with additional features, which will make my work much easier:
- for each memory allocation, I want to know the full call stack.
- moreover, I would like all memory leaks to be aggregated by the call stack.
- I don't want to do any change in my application.
- I may not even have the source code, and still want to discover memory leaks.
- the tool should be able to attach to any running application.
- I may want to activate a breakpoint each time a memory allocation is done from the "leaking" call stack, in order to attach the debugger in right location.
Well, I couldn't find any existing tool to fulfill all those requirements. Gradually I built such a tool for Windows applications, adding the mentioned capabilities. In the following sections of this article, you will learn how to use the tool, and will also find implementation notes, which will help you to understand the source code.
Using the tool
Examples of "leaking" applications
The demo archive includes two sample applications that generate memory leaks:
- VC++ console application: MemoryLeakExample.exe
Once started, it waits for the user's input. On each input it allocates three buffers, and fills them with some text, which includes an index of the current allocation. Of cause, the memory is not released.
- VB application: MemoryLeakExampleVB.exe
Each time you press Create New String button, a new string is allocated and added to the Collection, where it remains as a memory leak.
MemoryHooks VB wrapper
The VB wrapper is simple application with a very straightforward UI, which makes use of Remote invocation functions, exported by the
MemoryHooks DLL. You can see the snapshot on the top of this article.
- Run, for example, MemoryLeakExampleVB.exe.
- Make some allocations by pressing Create New String button number of times.
- Start MemoryHooksUI.exe.
- Type the ID of the running MemoryLeakExampleVB.exe process, and press the Attach button. For your convenience, the process ID is also shown in the caption of the MemoryLeakExampleVB.exe window.
- Press Start Monitoring button.
- Switch back to MemoryLeakExampleVB.exe, and allocate new string.
- Press Stop Monitoring Allocations button.
- Allocate new string in MemoryLeakExampleVB.exe.
- Press Stop Monitoring button.
- Press Report Leaks button. You can first change the name of the log file.
- View the log.
In step 10, you can also change the Number of stacks to report. Default value is
10, which means that only memory leaks from 10 "heaviest" call stacks will be printed in log (i.e. call stacks that generated biggest memory leaks at total).
If Include memory content option is checked, the content of un-released memory blocks will also be printed to the log.
By pressing Activate Breakpoints button, you instruct the program to select
10 stacks with the heaviest memory leaks, and stop on breakpoint next time any allocation is made from one of those stacks. You will see the same pop-up window, as when using
ASSERT, which will enable you to attach a debugger at the allocation point.
Of course, you can do the same with MemoryLeakExample.exe. Just instead of pressing Create New String button, type some data in the input prompt. You can see part of the final log in the second image in the top of this article.
Now watch what happens when you skip step 7, i.e. when you stop monitoring allocations and releases at the same time. You will find a new "memory leak" created by the C Run-Time, but actually this memory is released on next input.
You can also use
MemoryHooks as an imported library in other projects. The API is explained in details in the MemoryHooksAPI.h file, which comes with the source code.
There's also MemoryHooks_VB_API.txt file, which contains declarations of the same functions for VB.
Hooking memory allocations
The idea it to track all system calls to
HeapFree. I chose Detours library created by Microsoft Research. You can find the detailed description here. In two words: it rewrites the first instructions of the hooked function in the memory with a jump to the new location.
Traversing the call stack
The code can be found in the
FillStackInfo function of StackInfoManager.cpp. It starts with getting the pointer to the beginning of the current frame, from the
EBP register. First word in the frame contains a pointer to the beginning of the lower frame, and the second word - the return address of the function, i.e. the next instruction which will be executed in the calling function, when the current function returns.
Attaching to running process
Different techniques are explained in detail in the "Three Ways to Inject Your Code into Another Process" article by Robert Kuster. I've implemented one of them in the CodeInjector library, which can be used separately from the MemoryHooks tool.
Its API enables execution of any function of any DLL in the target process. The code runs in a separate thread. Following are the exported functions:
CODEINJECTOR_API BOOL ExecuteInRemoteProcess( DWORD dwPID,
LPCTSTR strLibrary, LPCTSTR strFunction )&
CODEINJECTOR_API BOOL LaunchInRemoteProcess( DWORD dwPID,
LPCTSTR strLibrary, LPCTSTR strFunction )&
- The solution only monitors
HealFree functions. In future I will handle
HeapFree function also, so the solution will work correctly also for applications which create and release heaps dynamically.
- Other types of allocations should be handles, such as Virtual Memory functions.
- You can't monitor two processes at the same time - the mechanism of inter-process communication should be redesigned.
Continuing this line of thinking further, we can talk about additional types of leaks. Actually for "cyclic" applications, all resources allocated in a single cycle should be released when it finishes. For example, all files that were opened must be closed etc.
- Detours library by Microsoft Research.
- "Three Ways to Inject Your Code into Another Process" article by Robert Kuster.