Introduction

Usually it is very cumbersome to detect and locate memory leaks. This article will provide a way to locate memory leaks in .Net applications. First I will talk about Resource Allocation and Garbage Collection algorithm and then will talk about detecting leaks in .Net apps. Pay attention to bold text in code section.

BACKGROUND

Resource Allocation

The CLR (Common Language Runtime) allocates all the resources on managed heap and releases them when they are no longer required by the application. C/C++ applications were prone to memory leaks because programmers have to manually allocate and free memory.

Runtime maintains a NextObjPtr for the next free space on the heap. When a new process is initialized, CLR allocates a contiguous space on the heap for this process represented by NextObjPtr and increments the NextObjPtr pointer to next free space. The space is contigous and is different from C++ heap where it is maintained as a linked list. GC heap is efficient as compared to C++ heap because to allocate new memory GC doesn't have to search from a list of free memory or through a linked list. As the time pass by, gaps start to appear in the heap as the objects get deleted and GC has to compact the heap which is costly. GC in .Net uses Win32 API VirtualAlloc or VirtualAllocEX to reserves memory.

.Net uses several types of memory like stack, unmanaged heap and the managed heap.

Stack : It is managed on a per thread basis and is used to store local variables, method parmaters and temporary valu<code>es. GC doesn't clean stack as its get automatically cleaned when the method returns. The refrences to objects is stored on the stack but the actual object gets allocated on heap and GC is aware of that. When the GC could not find a refrences for an object it removes it from the heap.

Unmanaged Heap: Unmanaged code will allocate objects on the unmanaged heap or stack. Managed code can also allocate object on unmanaged heap by calling Win32 APIs.

Managed Heap: Managed code allocates objects on managed heap and GC takes care of management of managed heap. GC also maintains a Large Object Heap to compensate the cost of moving large objects in memory.

Garbage Collection Algorithm

The garbage collector checks the heap for objects which are no longer used by application. If such objects exist then the GC removes those objects from the heap. Now the question is how GC finds out about these objects which are not used by application. Every application maintains a set of roots. Roots are like pointers to the objects on heap. All the global and static object pointers are considered as application roots. Any local variable on thread stack is considered as application root. This list of roots is maintained by JIT compiler and CLR and is made available to GC.

When the GC starts running, it treats all objects as garbage and makes an assumption that none of the objects on heap are accessible. It then starts walking with the list of application roots and starts building a graph of accessible objects. It marks all the objects on heap accessible if the objects are directly accessible as application root or indirectly accessible via any other object. For each application, the GC maintains a tree of references that tracks the objects referenced by the application. Using this approach, GC builds a list of live objects and then walks through the heap in search of objects which are not present in this list of live objects. After finding out the objects which are not present in this list of live objects, it marks them all as garbage and starts to compact the memory to remove holes which were created by unreferenced (dead) objects. It uses memcpy function to move the objects from one memory location to other and modifies the application roots to point to new locations.

If there is a live reference to the object then it is said to be strongly rooted. .Net also has a concept of Weak Reference. Any object can be created as a weak reference which tells GC that we want to access this object but if the GC is going through garbage collection then it can collect it. Weak reference is generally used for very large object which is easy to create but is costly to maintain in memory.

Moving objects in memory gives a significant performance hit. To improve the performance, GC does several optimizations like large object heap and Generations. Objects which are in size greater than 85,000 bytes are allocated on large object heap. Moving large objects in memory is costly so GC maintans a seprate heap for large objects which it never compacts. GC also maintains generations of objects. Whenever a new object is to be allocated and the managed heap doesn't have enough memory for the new object, a GC collection is performed. For the first time, every object in heap is considered in Gen 0. After that GC performs a collection the objects which survive are moved to Gen 1 and similarly which survives Gen 1 collection move to Gen 2. GC makes the assumption that a new object will have a short lifetime and a old object will have a longer lifetime. Whenever new memory is required, GC tries to collect memory from Gen 0 and if enough memory can't be obtained from Gen 0 collections then a Gen 1 or even Gen 2 collection is performed.

GC Sequence

These steps occur during each GC Collection

1. Execution Engine Suspension – The EE is suspended until all managed thread have reached a point in their code execution deemed "safe"

2. Mark – Objects that don't have roots are marked garbage

3. Plan – GC creates a budget for each generation being collected and then determines the amount of fragementation that will exist in the managed heap as a result of GC collection

4. Sweep – Deletes all objects marked for deletion

5. Compact – Move all non-pinned objects that survived GC to the lower end of heap

6. Execution Engine Restart – Restart the execution of managed threads

Roots in GC

There are different kinds of roots in GC

1. Strong Reference – If a strong reference exists for an object then it is considered in use and is not collected during the next GC collection.

2. Weak Reference – This is also a reference but the object can live till the next GC collection. Weak Reference works like a cache for object.

Finalization

GC can track the unmanaged resources lifetime but it can't reclaim the memory used by the resources unless destructors are used or code is written to override the Finalize in base class.

A finalizer exists to allow the programmer to clean up the native resources used by the object before the object is garbage collected. But using finalizer promotes the object collection to next generation. Whenever a new object with a Finalize method is allocated on Heap, a pointer to that object is placed on Finalization queue. During the garbage collection if the GC finds that an object is not reachable, it then searches the Finalization queue for any reference to the object. If it finds a reference then it removes the object from finalization queue and append it to other data structure called Freachable queue. At this point, Garbage Collector has finished identifyin garbage and compacts the memory. After that finalization thread empties the Freachable queue by executing each objects Finalize method. The next time GC collection is performed, GC sees this object as garbage and reclaims the memory assigned to this object.

It takes more time to reclaim the memory from objects having Finalize methods and affects performance so a Finalize method should only be used when required.

Detecting Memory Leak

Memory leak can occur either in stack, unmanaged heap or managed heap. There are many ways to find out that the memory is leaking like memory is increasing in Task Manager. Before start correcting the memory problem, you need to determine the kind of memory which is leaking. Perfmon can be used to examine the counters such as Process/Private bytes, .Net CLR Memory/# bytes in All heaps and the .Net CLR LocksAndThreads/# of the current logical thread. If the .Net CLR LocksAndThreads/# is increasing unexpectedly then the thread stack is leaking. If only Process/Private bytes are increasing but the .Net CLR Memory is not increasing then unmanaged memory is leaking else if both are increasing then the managed memory is leaking.

Figure 1 : Perfmon output for Privates Bytes and #Bytes in all Heaps

Stack Memory

Stack memory gets reclaimed after the method return. Stack memory can get leaked in two ways first that the method call consumes a significant amount of stack resources that never return, thereby never releasing the associated stack frame. The other is by creating background threads and never terminating them thus leaking the thread stack.

Unmanaged Heap Memory

If the total memory usage is increasing but the .Net CLR memory is not increasing then unmanaged memory is leaking. Unmanaged memory can leak in several ways. If the managed code is interoperating with unmanaged code and a leak exist in the unmanaged code. .Net doesn't make any guarantee that the finalizer for each object will get called. In the current implementation .Net has one finalizer thread. If there exist a finalizer which block this thread then the other finalizer will never get called and the unmanaged memory will leak which was supposed to be released. When an AppDomain is torn down, CLR tries to run all the finalizers but if a blocking finalizer exists then it can prevent the CLR from completing the AppDomain tear down. To prevent this, CLR implements a timeout on the process after which it stops the finalization process and the unmanaged memory which was supposed to be removed, is left leaked.

Managed Heap Memory

Managed memory can also get leaked by several ways like fragmentation of Large Object Heap. The memory in Large Object Heap never gets compacted so there is a loss in memory over there. Also, if there exists some objects which are not needed but there exist a reference to the objects then GC never claims the memory assigned to these objects.

This kind of leak is common and can be resolved using SOS.dll.

There are two ways to use SOS.dll .

1. WinDbg tool: This tool can be downloaded from
http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx

Run the application which you want to debug for memory problem. Start WinDbg tool and attach it to the application process.

OR

1. Open the application in Visual Studio 2005. Go to the properties of the project. In Debug tab, make sure you have "Enable unmanaged code Debugging" checked or if you are attaching to a process then in the "Attach to Process" window, click Select and then select "Managed code" and "Native code". Run the application and set a breakpoint in the code somewhere you want your application to break. Hit the breakpoint. Go to Debug -> Windows -> Immediate.

2. Run .load SOS.dll

SOS.dll is the most popular debugging extension used to debug the managed code. It has many powerful commands that can obtain information such as managed call stack, details about managed heap, objects in heap and much more.

3. Run !dumpheap –stat or !dumpheap –type PolicyEditor

This command scans the GC heaps and lists the objects that lie therein. –Stat argument is used to display the output to a statistical summary. More information about this (any) command can be found by !help dumpheap

The output will be like

03f354ec       20         1120 View.Console.Configuration.Workspace
03713e44       35         1120 System.Windows.Forms.LayoutEventHandler
03710bec       56         1120 System.ComponentModel.Container+Site
024dc014       70         1120 System.Configuration.PropertySourceInfo
0492ed48       94         1128 Infragistics.Shared.SubObjectBase+NotifyId
081d9ac4        2         1136 View.Manger.UI.PolicyEditor
03d79898        1         1140 System.Text.RegularExpressions.RegexCharClass+LowerCaseMapping[]
08857bdc       22         1144 Infragistics.Win.UltraWinToolbars.ComboBoxTool+ComboBoxToolExtraSharedProps
0869c604        8         1152 Infragistics.Win.UltraWinEditors.UltraTextEditorUIElement

4. Above command will list all the objects present in the memory. If you think that an object should not be present in memory and should have been garbage collected then open a Find (Ctrl +F) window and type the name of the object and search for it. If you cannot find the object then either it has been garbage collected or it was not instantiated. If you find the object for example the PolicyEditor object in the above list, then copy the MT (Method Table) address which is 081d9ac4 in the above case.

5. Run !dumpheap -mt 081d9ac4

This will list all the objects with this MethodTable address. Those objects are the instances of PolicyEditor.

The output will be something like

 ------------------------------
Heap 0
 Address       MT     Size
total 0 objects
------------------------------
Heap 1
 Address       MT     Size
total 0 objects
------------------------------
Heap 2
 Address       MT     Size
28404fd8 081d9ac4      568     
total 1 objects
------------------------------
Heap 3
 Address       MT     Size
2c49f098 081d9ac4      568     
total 1 objects
------------------------------
total 2 objects
Statistics:
      MT    Count    TotalSize Class Name
081d9ac4        2         1136 View.Manger.UI.PolicyEditor
Total 2 objects 
 

Copy the address of any instances for ex 28404fd8.

6. Run !gcroot 28404fd8

This will list the path to this object from the root of the GC tree. If some path exists for this object then this is considered non garbage and is not collected by GC which could be a reason for memory leakage.

The output will be like

Error during command: warning! Extension is using a feature which Visual does not implement.
Scan Thread 6460 OSTHread 193c
Scan Thread 1884 OSTHread 75c
Scan Thread 7520 OSTHread 1d60
Scan Thread 7716 OSTHread 1e24
Scan Thread 0 OSTHread 0
Scan Thread 7428 OSTHread 1d04
Scan Thread 0 OSTHread 0
Scan Thread 4728 OSTHread 1278

DOMAIN(01D637C8):HANDLE(Pinned):22b148c:Root:302d9250(System.Object[])->
2908633c(System.EventHandler)->
29085f2c(System.Object[])->
2c53a4cc(System.EventHandler)->
2c53a340(Infragistics.Win.UltraWinStatusBar.UltraStatusBar)->
2c5513b0(System.Collections.Hashtable)->
2c5513e8(System.Collections.Hashtable+bucket[])->
2c5390ac(Infragistics.Win.Printing.UltraPrintPreviewControl)->
2c530668(Infragistics.Win.UltraWinGrid.UltraGridPrintDocument)->
2c530858(System.Drawing.Printing.PrintEventHandler)->
2c52eff0(View.Windows.InfragisticsControls.CommonUltraGrid)->
24487ed0(System.EventHandler)->
2c52df64(View.Manger.UI.PolicyClarificationBrowser)->
2448f478(System.EventHandler)->
2c4e3714(View.Manger.UI.PolicyEditor)

DOMAIN(01D637C8):HANDLE(Pinned):22c12f8:Root:302d52b8(System.Object[])->
282e427c(System.Collections.Generic.Dictionary`2[[System.Object, mscorlib],[System.Collections.Generic.List`1[[Microsoft.Win32.SystemEvents+SystemEventInvokeInfo, System]], mscorlib]])->
282e4444(System.Collections.Generic.Dictionary`2+Entry[[System.Object, mscorlib],[System.Collections.Generic.List`1[[Microsoft.Win32.SystemEvents+SystemEventInvokeInfo, System]], mscorlib]][])->
282ee76c(System.Collections.Generic.List`1[[Microsoft.Win32.SystemEvents+SystemEventInvokeInfo, System]])->
2048e9c4(System.Object[])->
2c500534(Microsoft.Win32.SystemEvents+SystemEventInvokeInfo)->
2c500514(Microsoft.Win32.UserPreferenceChangedEventHandler)->
2c4fff54(Infragistics.Win.UltraWinStatusBar.UltraStatusBar)->
2446c824(System.Collections.Hashtable)->
2446c85c(System.Collections.Hashtable+bucket[])->
2c4fecc0(Infragistics.Win.Printing.UltraPrintPreviewControl)->
2c4f6a24(Infragistics.Win.UltraWinGrid.UltraGridPrintDocument)->
2c4f6c14(System.Drawing.Printing.PrintEventHandler)->
2c4f53ac(View.Windows.InfragisticsControls.CommonUltraGrid)->
2446e90c(System.EventHandler)->
24450e80(View.Manger.UI.EvidenceDefinitionMiniBrowser)->
2c4e34bc(View.Manger.Views.PolicyEditController)->
2c4e3524(View.Manger.Views.PolicyEdit)->
2449cfe0(System.Windows.Forms.LayoutEventArgs)->
2c4e3714(View.Manger.UI.PolicyEditor)

DOMAIN(01D637C8):HANDLE(WeakLn):22c1d88:Root:24496d2c(System.Windows.Forms.NativeMethods+WndProc)->
2c4e39fc(System.Windows.Forms.Control+ControlNativeWindow)->
2c4e3714(View.Manger.UI.PolicyEditor)
DOMAIN(01D637C8):HANDLE(WeakSh):22e313c:Root:2c4e39fc(System.Windows.Forms.Control+ControlNativeWindow)

7. In the above output we can see that PolicyClarificationBrowser object has an eventhandler which holds a reference to PolicyEditor. To find out who is hooking up this event, take the address of eventhandler which is 2448f478 and dump this object.

8. !dumpobj 2448f478

Name: System.EventHandler
MethodTable: 7910d61c
EEClass: 790c3a7c
Size: 32(0x20) bytes
 (C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
Fields:
      MT    Field   Offset                 Type VT     Attr    Value Name
790f9c18  40000f9        4        System.Object  0 instance 2c4e3714 _target
79109208  40000fa        8 ...ection.MethodBase  0 instance 00000000 _methodBase
790fe160  40000fb        c        System.IntPtr  0 instance 140144060 _methodPtr
790fe160  40000fc       10        System.IntPtr  0 instance        0 _methodPtrAux
790f9c18  4000106       14        System.Object  0 instance 00000000 _invocationList
790fe160  4000107       18        System.IntPtr  0 instance        0 _invocationCount



 

9. sd. If you take the address of the target in the
above output which is 2c4e3714 and run a !dumpobj with this
address then you can see in the output that it is of type PolicyEditor. To get the method that is hooked up as a handler,
convert the int value in _methodPtr to hex

?0n140144060 (This command will not work in the Immediate Window. To run this command you need to attach WinDbg to this or any other managed process or use google to do that)

The output will be

Evaluate expression: 140144060 = 085a6dbc

And then run !ip2md 085a6dbc

Failed to request MethodData, not in JIT code range

Sometimes this works and will give you the name of the method but if doesn't work then we can dump the object and then find out the method.

10. !dumpobj 2c4e3714

The output will be

Name: View.Manger.UI.PolicyEditor
MethodTable: 0820865c
EEClass: 08219fd8
Size: 568(0x238) bytes    
 (C:\Documents and Settings\testuser\Application Data\View\SMC-D-44725-B\View.Manger.UI.dll)
Fields:
      MT    Field   Offset                 Type VT     Attr    Value Name
790f9c18  4000184        4        System.Object  0 instance 00000000 __identity
024c1798  40008bc        8 ...ponentModel.ISite  0 instance 00000000 site
036f9c3c  40008bd        c ....EventHandlerList  0 instance 2448f8ac events
790f9c18  40008bb      108        System.Object  0   static 242e1afc EventDisposed

11. !dumpmt -md 0820865c

The output will be

EEClass: 08219fd8
Module: 036e48d8
Name: View.Manger.UI.PolicyEditor
mdToken: 02000069  (C:\Documents and Settings\testuser\Application Data\View\SMC-D-44725-B\View.Manger.UI.dll)
BaseSize: 0x238
ComponentSize: 0x0
Number of IFaces in IFaceMap: 15
Slots in VTable: 430
--------------------------------------
MethodDesc Table
   Entry MethodDesc      JIT Name
022a9fc5   022a9ec8     NONE System.ComponentModel.Component.ToString()
793539c0   7913bd50   PreJIT System.Object.Equals(System.Object)
…
…
085a6d44   08208578     NONE View.Manger.UI.PolicyEditor.btnStatementDelete_Click(System.Object, System.EventArgs)
085a6de4   08208580     NONE View.Manger.UI.PolicyEditor.btnProperties_Click(System.Object, System.EventArgs)
085a6dbc   08208588     NONE View.Manger.UI.PolicyEditor.ClarificationBrowser_OpenClick(System.Object, System.EventArgs)
08208f0d   08208590     NONE View.Manger.UI.PolicyEditor.OpenClarification()
085a6dd0   08208598     NONE View.Manger.UI.PolicyEditor.Clarification_SelectionChanged(System.Object, View.Manger.UI.SelectEventArgs)
085a6d94   082085a0     NONE View.Manger.UI.PolicyEditor.btnExceptionNew_Click(System.Object, System.EventArgs)

…
…

In the above output search for the hex address we calculated above from _methodptr and you will get the name of the method which is still holding a reference for PolicyEditor object and is the reason for Memory Leak.

Points of Interest

History

Initially posted on Jul 3, 2007.