Click here to Skip to main content
15,867,568 members
Articles / Programming Languages / Visual Basic
Article

Easy Detection of Memory Leaks

Rate me:
Please Sign up or sign in to vote.
4.73/5 (25 votes)
4 Aug 20058 min read 185.4K   8.6K   122   48
Describes a tool for easy detection of memory leaks.

Memory Hooks UI

Memory leaks log

Contents

Introduction

Memory Hooks is a tool for easy detection of memory leaks in any Windows application. Main features:

  • No modifications in source code. Actually the code is not required.
  • Works for any Windows application written in any language.
  • Attaches to any running process.
  • Especially effective for applications working in cyclic patterns.
  • Aggregation of memory leaks by call stack.
  • Inserting breakpoints for easy debugging.

Memory Hooks can be used as a library, or as stand-alone tool using the provided VB wrapper.

Memory Leaks - Different Approach

What is "memory leak"? Usually we mean "a block of memory that was allocated by the program, and was not released". To be more precise we should add: "and was not released before the program ended".

I think this definition is not appropriate for many applications. First let's remember that when a program ends, all the memory that was allocated by it is automatically released by the OS. So it will not make much difference if the memory was released by the process just before the termination. The more interesting parameter, especially for long-running programs, is the change in memory consumption in time. As a simple example, we can look at programs that work in "cyclic" patterns. Here cycle refers to a unit of work, after which the process should return to the state in which it was before the cycle started. For instance: a text editor which opens and works with a document, and then closes it and a service which processes a single request. In all those cases we can talk about "memory leak per cycle". In case such memory leaks exist, even if relatively small, it can cause serious performance problems over time.

Conclusions that we may get with this approach:

  • It is more important to avoid cyclic memory leaks, than one-time leaks - O(n) VS O(1).
  • Such memory leaks may exist even in programs written in languages featuring automatic garbage collection. For example, in VB you can continuously create new objects, and add them to a Collection, so you will always have a live reference to an object.

Memory leaks detection process

Prior to selecting the right tool for hunting cyclic memory leaks, let's think what additional important characteristics we would like it to have. We'll take a text editor "my_edit.exe" application as an example. I would like to perform the following steps:

  1. run "my_edit.exe".
  2. open existing document, add new line, save changes and close the document.
  3. start monitoring memory allocations.
  4. repeat step 2.
  5. stop monitoring memory allocations.
  6. repeat step 2.
  7. report: all memory allocations done in step 6, which was not released in steps 6-8.

Here I would like to explain some of the steps in more details:

  • Step 2: before starting memory monitoring, I want to make sure the application is fully initialized. It's possible that there will be some one-time allocations while opening the first document.
  • Step 6: here I let the application a chance to do full cleanup of memory allocated in step 4. There may be, for example, some objects remaining in memory after step 4, which will be released only when the next document is processed.

Required features - a wish list

Given scenario implies the following features of the memory monitoring tool:

  1. it should be able to begin the monitoring at some point in time.
  2. it should be able to stop registering new allocations, but still watch memory being released.

I can extend this "wish list" with additional features, which will make my work much easier:

  1. for each memory allocation, I want to know the full call stack.
  2. moreover, I would like all memory leaks to be aggregated by the call stack.
  3. I don't want to do any change in my application.
  4. I may not even have the source code, and still want to discover memory leaks.
  5. the tool should be able to attach to any running application.
  6. I may want to activate a breakpoint each time a memory allocation is done from the "leaking" call stack, in order to attach the debugger in right location.

Well, I couldn't find any existing tool to fulfill all those requirements. Gradually I built such a tool for Windows applications, adding the mentioned capabilities. In the following sections of this article, you will learn how to use the tool, and will also find implementation notes, which will help you to understand the source code.

Using the tool

Examples of "leaking" applications

The demo archive includes two sample applications that generate memory leaks:

  1. VC++ console application: MemoryLeakExample.exe

    Once started, it waits for the user's input. On each input it allocates three buffers, and fills them with some text, which includes an index of the current allocation. Of cause, the memory is not released.

  2. VB application: MemoryLeakExampleVB.exe

    Each time you press Create New String button, a new string is allocated and added to the Collection, where it remains as a memory leak.

MemoryHooks VB wrapper

The VB wrapper is simple application with a very straightforward UI, which makes use of Remote invocation functions, exported by the MemoryHooks DLL. You can see the snapshot on the top of this article.

Step-by-step example:

  1. Run, for example, MemoryLeakExampleVB.exe.
  2. Make some allocations by pressing Create New String button number of times.
  3. Start MemoryHooksUI.exe.
  4. Type the ID of the running MemoryLeakExampleVB.exe process, and press the Attach button. For your convenience, the process ID is also shown in the caption of the MemoryLeakExampleVB.exe window.
  5. Press Start Monitoring button.
  6. Switch back to MemoryLeakExampleVB.exe, and allocate new string.
  7. Press Stop Monitoring Allocations button.
  8. Allocate new string in MemoryLeakExampleVB.exe.
  9. Press Stop Monitoring button.
  10. Press Report Leaks button. You can first change the name of the log file.
  11. View the log.

In step 10, you can also change the Number of stacks to report. Default value is 10, which means that only memory leaks from 10 "heaviest" call stacks will be printed in log (i.e. call stacks that generated biggest memory leaks at total).

If Include memory content option is checked, the content of un-released memory blocks will also be printed to the log.

By pressing Activate Breakpoints button, you instruct the program to select 10 stacks with the heaviest memory leaks, and stop on breakpoint next time any allocation is made from one of those stacks. You will see the same pop-up window, as when using ASSERT, which will enable you to attach a debugger at the allocation point.

Interesting observation

Of course, you can do the same with MemoryLeakExample.exe. Just instead of pressing Create New String button, type some data in the input prompt. You can see part of the final log in the second image in the top of this article.

Now watch what happens when you skip step 7, i.e. when you stop monitoring allocations and releases at the same time. You will find a new "memory leak" created by the C Run-Time, but actually this memory is released on next input.

MemoryHooks DLL

You can also use MemoryHooks as an imported library in other projects. The API is explained in details in the MemoryHooksAPI.h file, which comes with the source code.

There's also MemoryHooks_VB_API.txt file, which contains declarations of the same functions for VB.

Implementation notes

Hooking memory allocations

The idea it to track all system calls to HealAlloc/HeapFree. I chose Detours library created by Microsoft Research. You can find the detailed description here. In two words: it rewrites the first instructions of the hooked function in the memory with a jump to the new location.

Traversing the call stack

The code can be found in the FillStackInfo function of StackInfoManager.cpp. It starts with getting the pointer to the beginning of the current frame, from the EBP register. First word in the frame contains a pointer to the beginning of the lower frame, and the second word - the return address of the function, i.e. the next instruction which will be executed in the calling function, when the current function returns.

Attaching to running process

Different techniques are explained in detail in the "Three Ways to Inject Your Code into Another Process" article by Robert Kuster. I've implemented one of them in the CodeInjector library, which can be used separately from the MemoryHooks tool.

Its API enables execution of any function of any DLL in the target process. The code runs in a separate thread. Following are the exported functions:

// Synchronic: waits until the operation is finished
CODEINJECTOR_API BOOL ExecuteInRemoteProcess( DWORD dwPID, 
            LPCTSTR strLibrary, LPCTSTR strFunction )&

// A-synchronic: launches the code on remote process, and returns
CODEINJECTOR_API BOOL LaunchInRemoteProcess( DWORD dwPID, 
           LPCTSTR strLibrary, LPCTSTR strFunction )&

Limitations

  • The solution only monitors HeapAlloc, HealRealloc and HealFree functions. In future I will handle HeapFree function also, so the solution will work correctly also for applications which create and release heaps dynamically.
  • Other types of allocations should be handles, such as Virtual Memory functions.
  • You can't monitor two processes at the same time - the mechanism of inter-process communication should be redesigned.

Next steps

Continuing this line of thinking further, we can talk about additional types of leaks. Actually for "cyclic" applications, all resources allocated in a single cycle should be released when it finishes. For example, all files that were opened must be closed etc.

References:

  1. Detours library by Microsoft Research.
  2. "Three Ways to Inject Your Code into Another Process" article by Robert Kuster.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


Written By
Web Developer
Israel Israel
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
GeneralStart Monitoring Button is not enabled in windows Pin
Member 1341887920-Sep-17 1:45
Member 1341887920-Sep-17 1:45 
PraiseI just want to say: Thank You. Pin
FloatinWind7-Apr-17 23:55
FloatinWind7-Apr-17 23:55 
QuestionRe-write this solution from zero... Pin
Tarmo Pikaro9-Feb-16 9:48
Tarmo Pikaro9-Feb-16 9:48 
Question64-bit architecture not supported ? Pin
Tarmo Pikaro23-Nov-15 20:54
Tarmo Pikaro23-Nov-15 20:54 
Questioncomdlg.ocx missing Pin
Member 917126812-Feb-15 4:33
Member 917126812-Feb-15 4:33 
BugBug (Attach button does nothing) Pin
epuskar19-Mar-13 5:32
epuskar19-Mar-13 5:32 
GeneralVery good work ! Only a little bug to submit Pin
slimfast200110-Apr-10 0:04
slimfast200110-Apr-10 0:04 
GeneralRe: Very good work ! Only a little bug to submit Pin
Michael Gopshtein11-Apr-10 4:06
Michael Gopshtein11-Apr-10 4:06 
GeneralSimple, and very useful! Nice! Pin
Abu Mami23-Jan-09 0:35
Abu Mami23-Jan-09 0:35 
GeneralRe: Simple, and very useful! Nice! Pin
Michael Gopshtein23-Jan-09 3:30
Michael Gopshtein23-Jan-09 3:30 
GeneralRegarding Detour Pin
Saki7865-Jul-07 20:35
Saki7865-Jul-07 20:35 
GeneralRe: Regarding Detour Pin
Michael Gopshtein5-Jul-07 20:46
Michael Gopshtein5-Jul-07 20:46 
GeneralEncapsulating it with VC Pin
xyecloudy19-May-07 2:17
xyecloudy19-May-07 2:17 
GeneralRe: Encapsulating it with VC Pin
Michael Gopshtein19-May-07 5:38
Michael Gopshtein19-May-07 5:38 
QuestionGetting "File Not Found" error in NT 4 (SP6) Pin
nickbeccia3-Jan-07 7:56
nickbeccia3-Jan-07 7:56 
AnswerRe: Getting "File Not Found" error in NT 4 (SP6) Pin
Michael Gopshtein3-Jan-07 19:11
Michael Gopshtein3-Jan-07 19:11 
Hi Nick,

I never tried it on NT4, and don't know about any related issues. The error seems to happen when VB tries to load a DLL, so my guess is that it's one of the following:
1) It can't find the file
2) It can't load the DLL

Solutions:
1) Make sure you've copied all files, and put them all in same folder with VB program.
2) The DLL may be incompatible with NT4, or some dependencies may be missing (both of VB wrapper, and of other DLLs). If you have a Visual Studio on NT4 box, you can just compile the source files, and see what's wrong. You can also open the DLLs with "Dependency Walker" (part of VS, you can do it also on 2K box), and see the list of required DLLs.

If you need help with the last part (finding dependencies), I can do it on my compuer.

Michael

GeneralRe: Getting "File Not Found" error in NT 4 (SP6) Pin
nickbeccia4-Jan-07 6:02
nickbeccia4-Jan-07 6:02 
QuestionIncorrect Stack Info [modified] Pin
muldragon4-Aug-06 0:47
muldragon4-Aug-06 0:47 
AnswerRe: Incorrect Stack Info Pin
Michael Gopshtein5-Aug-06 9:17
Michael Gopshtein5-Aug-06 9:17 
GeneralCom Addin Pin
HakunaMatada17-Mar-06 19:42
HakunaMatada17-Mar-06 19:42 
GeneralRe: Com Addin Pin
Michael Gopshtein18-Mar-06 13:18
Michael Gopshtein18-Mar-06 13:18 
GeneralRe: Com Addin Pin
HakunaMatada20-Mar-06 17:37
HakunaMatada20-Mar-06 17:37 
GeneralRe: Com Addin Pin
Michael Gopshtein20-Mar-06 18:22
Michael Gopshtein20-Mar-06 18:22 
GeneralRe: Com Addin Pin
HakunaMatada22-Mar-06 2:21
HakunaMatada22-Mar-06 2:21 
GeneralRe: Com Addin Pin
Michael Gopshtein22-Mar-06 5:13
Michael Gopshtein22-Mar-06 5:13 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.