Check for and free memory leaks

• Download DLL, lib and source - 58.2 Kb
• Download MSVC project with test program - 22.9 Kb

Introduction

This code snippet can be used to detect and free memory leaks in C programs. It isn't suitable for C++ in its present form. It is fast and introduces little overhead in the way of extra memory, especially if thread contention is rare. This means it is suitable for use in release builds to auto free memory leaks in C DLLs and programs that are written by others. This avoids the problem of possibly introducing new bugs as a result of editing the code to fix the leak.

In diagnostic mode it reports the line number and file where the leak occurred. This adds an extra 22 bytes on average to each memory allocation. Since MSVC can do this already, and probably most compilers, I suppose the main advantage here is that you can customize the code, - also you can use it with release builds to test them for memory leaks too, in case where the release differs significantly from the debug build.

In auto freemode, you can call it to free any memory allocated, that got allocated. In this case it adds 10 bytes on average, to each memory allocation. This is probably most useful for a DLL, - if the allocations are done separately for each call of a routine in the DLL, you could set it to auto free memory when the routine in the DLL returns.

It is implemented for malloc, realloc, calloc, and strdup. It would be easy to extend it to the rarer memory allocation routines such as _expand, and the wide string versions of strdup.

Note

You don't want to mix this library with ordinary uses of malloc and free because when it gets given a pointer to free, the first thing it does is to decrement the pointer and access the decremented pointer to find the record number (this technique is what makes it so fast). This will cause an access violation if the pointer was originally allocated by normal use of malloc. The other way round, if you try to free one of its pointers using normal free then because the pointers are incremented before they are passed to the calling app, you will get errors there too.

Every allocation made using this library needs to be freed using it, and it should only be used to free memory that it allocated itself. However, you can use it for a particular sections of your source code by including its header for those sections only. Alternatively, define code_uses_redefined_malloc and then replace malloc by _malloc_, calloc by _calloc_, free by _free_ and strdup by _strdup_ throughout the section where you want to use it.

Also be sure to note - it is for pure C programs which rely on use of malloc etc. for memory allocation. It doesn't call Dtors, nor is it appropriate for use with new.

Background

Memory leaks are an issue in programs and dlls that run 24/7. This library may be useful to auto free memory in such cases. It could be useful particularly if the dll is a conversion of a program that was originally written with reliance on the operating system to auto free it when the program exited. You just need to include the header and there is no need to edit the source code and potentially introduce new bugs. It can also be used for memory leak diagnostics to locate the places where the leaks occurred.

Using the code

How to use it - general

You just use your memory allocation routines as usual - and then when you have freed all the memory you are going to free, call MemLeakCheckAndFree()

This will show a diagnostics report if it found any pointers still unallocated - and will free them. If you switch off diagnostics then this routine is the one that does the auto free for you.

In diagnostic mode, the warnings are written to stderr as is useful for a console app. However if you are using it with a Windows program you would make your own callback routines to handle the warnings. Example callbacks, which you can use for this are included within comments in the header:

SetMemLeakWarnProc(MemLeakWarnProc);

and

SetMemLeakLogProc(MemLeakLogProc);

The log proc is used by MemLeakCheckAndFree since a potentially long list of warnings may need to be shown - typically one might write those to a file. Here is what its output might look like:

************************************************************
2 PM Saturday, July 12, 2003 GMT Daylight Time
************************************************************


Memory Leak!
File: I:\Lissajous\main.c, line 46, size 100
File: I:\Lissajous\main.c, line 248, size 1000

It also calls MemLeakWarnProc once at the end, to alert the user that there has been a memory leak.

To use it as a DLL

Link with the relevant DLL, either MemLeakCheckt.dll for diagnostics, or MemLeakCheckat.dll to auto free only (add the relevant .lib file to your linker options).

Now include the header after all your standard includes and before your own source code where the allocation and free occurs.

In the header, define

#define LINK_MEMLEAK_CHECK_AS_DLL

Then, if you are linking with MemLeakCheckt.dll (the one with the diagnostics), you need to define

#define cl_mem_leak_check 
#define cl_mem_leak_diagnostics

or for MemLeackCheckat.dll, just define the cl_mem_leak_check

Since you may very well use a different version of the C run time library from the DLL (whether single or multithreaded, release or debug), it might be an idea to add this code at the end of one of your source files after all the memory allocations, where the #undefs won't affect them. Then call it when your app starts, to set the memory allocation routines for the DLL to the ones for your CRT:

#ifdef cl_mem_leak_diagnostics

#undef malloc
#undef calloc
#undef realloc
#undef memcpy
#undef free
#include "malloc.h"

void init_memleakcheck_alloc(void)
{
 set_mlc_malloc(malloc);
 set_mlc_calloc(calloc);
 set_mlc_realloc(realloc);
 set_mlc_free(free);
 set_mlc_memcpy(memcpy);
 set_mlc_strlen(strlen);
 set_mlc_strcpy(strcpy);
}

#endif

I don't know if this step is necessary. It is needed in the case of file pointers as if you use a file pointer in a library that uses a different version of the CRT from the calling app, then it is no longer valid and will probably cause an access violation, and this method can fix the problem in that case. But the malloc and free pointers seem to work okay in the calling app. Even if you don't do it. I think it is probably safest though, to do it like this. Any information about this appreciated :-).

If you want to use it to auto free memory in a DLL each time one of the routines exits, then I think it is best to include it as source code and to use thread local variables - see next section.

To use it as source code as part of your project

Add the source code to your project, and add the header after all the includes and before uses of malloc etc. This header will redefine malloc, realloc, calloc, and strdup so that the versions in MemLeakCheck get used instead of the standard ones. These then in turn call the standard malloc, etc. to do the actual memory allocation.

As before, define:

#define cl_mem_leak_check 
#define cl_mem_leak_diagnostics

and make sure you comment out the two definitions that are needed to build it as a DLL or link to it as a DLL.

If you want to use it to auto free memory in a DLL each time one of the routines exits, then it is probably best to use thread local variables - in case several threads simultaneously want to use the same routine.

When you use thread local variables, each thread has to call the autofree routine separately. In the case where you free the memory every time a particular routine returns, then this is easy, as each time it will only free it for the particular thread it was called by.

To use thread local variables define this in the header:

#define MLC_USE_THREAD_LOCAL_VARIABLES

Points of interest

The idea is to use the first few bytes of each allocation to hold the information you want for the allocation - in diagnostic mode it has a pointer to the static string for the file name, and the line number. In both modes it also has a record number which is an index into an array of pointers.

Then when memory needs to be allocated, you allocate the memory needed plus a bit extra for this extra information, and return a pointer incremented to point at the data itself at the end of the record. When the memory needs to be freed, you decrement the pointer given to you by the calling app in order to find its record, and the record number.

The special trick used to make it fast is the addition of the record number here. When memory needs to be freed, one can go straight to the relevant entry in the array of pointers, with no need to look it up or do any kind of a search. Then when you free the memory, you set the entry to NULL to indicate that the pointer is no longer in play. From time to time you need to compact the array of pointers - just walk through it from the start and move all the pointers that are not null back, and access them and update their record numbers as you do so.

To deal with thread contention, since this is expected to be rare, it uses InterlockedExchange in combination with an efficient busy wait (surrendering the rest of each time slice using Sleep(0) until access is obtained). This is very fast, just a single assembly language instruction to execute to get access to the section, so better than critical sections.

In the case where contention does occur frequently, then this means that one of your threads is repeatedly calling Sleep(0) while the other allocates memory. This shouldn't have too much of a performance impact - the repeatedly sleeping thread can't do anything until it gets the memory anyway. It shouldn't affect other programs that may be running at the same time either, as it only uses a few assembly language instructions per time slice. So - I can't see that this will have much performance impact - please correct me in this if needed and I'll be interested in information about how it performed in such a situation if anyone tries it for an app. with high thread contention for memory allocation. If it was found to have a performance impact, then one solution would be to use thread local variables, and do the auto free separately for each of the contending threads.

The memory and time requirements are both low. However from time to time it compacts the array of pointers by traversing the array. This involves inspecting each entry in an array of n pointers where n is at least 1000, and up to 1.5 times the number of allocations you have in use and not yet freed. So I think it isn't much of a time penalty in most situations.

The time needed to compact the array of pointers may be significant if one has tens or hundreds of thousands of small allocations in play at once, and if the app. is also highly time critical. In this case one might want to add a routine to the library in order to temporarily suspend the array compaction whenever one is doing time critical work, and resume it again at an appropriate later occasion (an easy mod). Perhaps there is little one can do to speed it up, since the idea of doing things this way was to make the allocation and frees, immediate with no need even for as much time penalty as is needed to insert an entry in a binary tree, which I think is more important than eliminating the array compaction step. Then to compact the array I hardly see how one can improve on a simple array traversal from the start, without adding to memory requirements or slowing the addition or removal of elements to it. Again please be sure to correct me if there is a way to improve on this - I'll be delighted to hear of any improvements.

For more information see the Mem Leak Check home page.

History

• Update
Updated the description to correct earlier version. I thought that memory needed to be freed whenever you exit from a program, but it seems windows does it automatically - see the discussion. Added a note about its use for just a particular section, as one might use it in 24/7 type dlls.
• First release