Click here to Skip to main content
Click here to Skip to main content
Go to top

Easy Detection of Memory Leaks

, 4 Aug 2005
Rate this:
Please Sign up or sign in to vote.
Describes a tool for easy detection of memory leaks.

Memory Hooks UI

Memory leaks log

Contents

Introduction

Memory Hooks is a tool for easy detection of memory leaks in any Windows application. Main features:

  • No modifications in source code. Actually the code is not required.
  • Works for any Windows application written in any language.
  • Attaches to any running process.
  • Especially effective for applications working in cyclic patterns.
  • Aggregation of memory leaks by call stack.
  • Inserting breakpoints for easy debugging.

Memory Hooks can be used as a library, or as stand-alone tool using the provided VB wrapper.

Memory Leaks - Different Approach

What is "memory leak"? Usually we mean "a block of memory that was allocated by the program, and was not released". To be more precise we should add: "and was not released before the program ended".

I think this definition is not appropriate for many applications. First let's remember that when a program ends, all the memory that was allocated by it is automatically released by the OS. So it will not make much difference if the memory was released by the process just before the termination. The more interesting parameter, especially for long-running programs, is the change in memory consumption in time. As a simple example, we can look at programs that work in "cyclic" patterns. Here cycle refers to a unit of work, after which the process should return to the state in which it was before the cycle started. For instance: a text editor which opens and works with a document, and then closes it and a service which processes a single request. In all those cases we can talk about "memory leak per cycle". In case such memory leaks exist, even if relatively small, it can cause serious performance problems over time.

Conclusions that we may get with this approach:

  • It is more important to avoid cyclic memory leaks, than one-time leaks - O(n) VS O(1).
  • Such memory leaks may exist even in programs written in languages featuring automatic garbage collection. For example, in VB you can continuously create new objects, and add them to a Collection, so you will always have a live reference to an object.

Memory leaks detection process

Prior to selecting the right tool for hunting cyclic memory leaks, let's think what additional important characteristics we would like it to have. We'll take a text editor "my_edit.exe" application as an example. I would like to perform the following steps:

  1. run "my_edit.exe".
  2. open existing document, add new line, save changes and close the document.
  3. start monitoring memory allocations.
  4. repeat step 2.
  5. stop monitoring memory allocations.
  6. repeat step 2.
  7. report: all memory allocations done in step 6, which was not released in steps 6-8.

Here I would like to explain some of the steps in more details:

  • Step 2: before starting memory monitoring, I want to make sure the application is fully initialized. It's possible that there will be some one-time allocations while opening the first document.
  • Step 6: here I let the application a chance to do full cleanup of memory allocated in step 4. There may be, for example, some objects remaining in memory after step 4, which will be released only when the next document is processed.

Required features - a wish list

Given scenario implies the following features of the memory monitoring tool:

  1. it should be able to begin the monitoring at some point in time.
  2. it should be able to stop registering new allocations, but still watch memory being released.

I can extend this "wish list" with additional features, which will make my work much easier:

  1. for each memory allocation, I want to know the full call stack.
  2. moreover, I would like all memory leaks to be aggregated by the call stack.
  3. I don't want to do any change in my application.
  4. I may not even have the source code, and still want to discover memory leaks.
  5. the tool should be able to attach to any running application.
  6. I may want to activate a breakpoint each time a memory allocation is done from the "leaking" call stack, in order to attach the debugger in right location.

Well, I couldn't find any existing tool to fulfill all those requirements. Gradually I built such a tool for Windows applications, adding the mentioned capabilities. In the following sections of this article, you will learn how to use the tool, and will also find implementation notes, which will help you to understand the source code.

Using the tool

Examples of "leaking" applications

The demo archive includes two sample applications that generate memory leaks:

  1. VC++ console application: MemoryLeakExample.exe

    Once started, it waits for the user's input. On each input it allocates three buffers, and fills them with some text, which includes an index of the current allocation. Of cause, the memory is not released.

  2. VB application: MemoryLeakExampleVB.exe

    Each time you press Create New String button, a new string is allocated and added to the Collection, where it remains as a memory leak.

MemoryHooks VB wrapper

The VB wrapper is simple application with a very straightforward UI, which makes use of Remote invocation functions, exported by the MemoryHooks DLL. You can see the snapshot on the top of this article.

Step-by-step example:

  1. Run, for example, MemoryLeakExampleVB.exe.
  2. Make some allocations by pressing Create New String button number of times.
  3. Start MemoryHooksUI.exe.
  4. Type the ID of the running MemoryLeakExampleVB.exe process, and press the Attach button. For your convenience, the process ID is also shown in the caption of the MemoryLeakExampleVB.exe window.
  5. Press Start Monitoring button.
  6. Switch back to MemoryLeakExampleVB.exe, and allocate new string.
  7. Press Stop Monitoring Allocations button.
  8. Allocate new string in MemoryLeakExampleVB.exe.
  9. Press Stop Monitoring button.
  10. Press Report Leaks button. You can first change the name of the log file.
  11. View the log.

In step 10, you can also change the Number of stacks to report. Default value is 10, which means that only memory leaks from 10 "heaviest" call stacks will be printed in log (i.e. call stacks that generated biggest memory leaks at total).

If Include memory content option is checked, the content of un-released memory blocks will also be printed to the log.

By pressing Activate Breakpoints button, you instruct the program to select 10 stacks with the heaviest memory leaks, and stop on breakpoint next time any allocation is made from one of those stacks. You will see the same pop-up window, as when using ASSERT, which will enable you to attach a debugger at the allocation point.

Interesting observation

Of course, you can do the same with MemoryLeakExample.exe. Just instead of pressing Create New String button, type some data in the input prompt. You can see part of the final log in the second image in the top of this article.

Now watch what happens when you skip step 7, i.e. when you stop monitoring allocations and releases at the same time. You will find a new "memory leak" created by the C Run-Time, but actually this memory is released on next input.

MemoryHooks DLL

You can also use MemoryHooks as an imported library in other projects. The API is explained in details in the MemoryHooksAPI.h file, which comes with the source code.

There's also MemoryHooks_VB_API.txt file, which contains declarations of the same functions for VB.

Implementation notes

Hooking memory allocations

The idea it to track all system calls to HealAlloc/HeapFree. I chose Detours library created by Microsoft Research. You can find the detailed description here. In two words: it rewrites the first instructions of the hooked function in the memory with a jump to the new location.

Traversing the call stack

The code can be found in the FillStackInfo function of StackInfoManager.cpp. It starts with getting the pointer to the beginning of the current frame, from the EBP register. First word in the frame contains a pointer to the beginning of the lower frame, and the second word - the return address of the function, i.e. the next instruction which will be executed in the calling function, when the current function returns.

Attaching to running process

Different techniques are explained in detail in the "Three Ways to Inject Your Code into Another Process" article by Robert Kuster. I've implemented one of them in the CodeInjector library, which can be used separately from the MemoryHooks tool.

Its API enables execution of any function of any DLL in the target process. The code runs in a separate thread. Following are the exported functions:

// Synchronic: waits until the operation is finished
CODEINJECTOR_API BOOL ExecuteInRemoteProcess( DWORD dwPID, 
            LPCTSTR strLibrary, LPCTSTR strFunction )&

// A-synchronic: launches the code on remote process, and returns
CODEINJECTOR_API BOOL LaunchInRemoteProcess( DWORD dwPID, 
           LPCTSTR strLibrary, LPCTSTR strFunction )&

Limitations

  • The solution only monitors HeapAlloc, HealRealloc and HealFree functions. In future I will handle HeapFree function also, so the solution will work correctly also for applications which create and release heaps dynamically.
  • Other types of allocations should be handles, such as Virtual Memory functions.
  • You can't monitor two processes at the same time - the mechanism of inter-process communication should be redesigned.

Next steps

Continuing this line of thinking further, we can talk about additional types of leaks. Actually for "cyclic" applications, all resources allocated in a single cycle should be released when it finishes. For example, all files that were opened must be closed etc.

References:

  1. Detours library by Microsoft Research.
  2. "Three Ways to Inject Your Code into Another Process" article by Robert Kuster.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

Michael Gopshtein
Web Developer
Israel Israel
No Biography provided

Comments and Discussions

 
BugBug (Attach button does nothing) Pinmemberepuskar19-Mar-13 5:32 
GeneralVery good work ! Only a little bug to submit Pinmemberslimfast200110-Apr-10 0:04 
GeneralRe: Very good work ! Only a little bug to submit PinmemberMichael Gopshtein11-Apr-10 4:06 
GeneralSimple, and very useful! Nice! PinmemberAbu Mami23-Jan-09 0:35 
GeneralRe: Simple, and very useful! Nice! PinmemberMichael Gopshtein23-Jan-09 3:30 
GeneralRegarding Detour PinmemberSaki7865-Jul-07 20:35 
GeneralRe: Regarding Detour PinmemberMichael Gopshtein5-Jul-07 20:46 
GeneralEncapsulating it with VC Pinmemberxyecloudy19-May-07 2:17 
GeneralRe: Encapsulating it with VC PinmemberMichael Gopshtein19-May-07 5:38 
QuestionGetting "File Not Found" error in NT 4 (SP6) Pinmembernickbeccia3-Jan-07 7:56 
AnswerRe: Getting "File Not Found" error in NT 4 (SP6) PinmemberMichael Gopshtein3-Jan-07 19:11 
GeneralRe: Getting "File Not Found" error in NT 4 (SP6) Pinmembernickbeccia4-Jan-07 6:02 
QuestionIncorrect Stack Info [modified] Pinmembermuldragon4-Aug-06 0:47 
AnswerRe: Incorrect Stack Info PinmemberMichael Gopshtein5-Aug-06 9:17 
GeneralCom Addin PinmemberSave My Soul - (SMS)17-Mar-06 19:42 
GeneralRe: Com Addin PinmemberMichael Gopshtein18-Mar-06 13:18 
GeneralRe: Com Addin PinmemberSave My Soul - (SMS)20-Mar-06 17:37 
GeneralRe: Com Addin PinmemberMichael Gopshtein20-Mar-06 18:22 
GeneralRe: Com Addin PinmemberSave My Soul - (SMS)22-Mar-06 2:21 
GeneralRe: Com Addin PinmemberMichael Gopshtein22-Mar-06 5:13 
GeneralRe: Com Addin PinmemberSave My Soul - (SMS)22-Mar-06 17:16 
Generalrequest plz open it Pinmemberdhawalagarwal14-Feb-06 20:23 
Generalcant seem to be able to dl the source Pinmembermail2liam12-Aug-05 5:35 
QuestionWhat does "create and release heaps dynamically" mean? Pinmembertingzheng11-Aug-05 22:18 
AnswerRe: What does "create and release heaps dynamically" mean? PinmemberMichael Gopshtein12-Aug-05 11:53 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140916.1 | Last Updated 5 Aug 2005
Article Copyright 2005 by Michael Gopshtein
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid