Click here to Skip to main content
15,867,453 members
Articles / Desktop Programming / Windows Forms

Mnemonic: Assisting Your (virtual) Memory

Rate me:
Please Sign up or sign in to vote.
4.84/5 (10 votes)
9 Feb 2012CPOL4 min read 36.1K   1K   19   11
A tool for visualizing the virtual memory used by Windows processes

Introduction

This article describes a tool I wrote to show, pictorially and dynamically, the consumption of virtual memory by a Windows process.

Background

Recently, I needed to investigate the way in which a Windows process was consuming Virtual Memory (VM). I wanted to get a picture in my head of the available VM and how it was being allocated, freed and mapped by the process, and phenomena such as virtual memory fragmentation.

I came across this tool by Charles Bailey: http://hashpling.org/asm/. It works well and it helped, but I wanted more information about particular types of allocation (those corresponding to memory-mapped files and managed and unmanaged assemblies loaded by the process), and I wanted to be able to understand for myself what was going on 'under the hood'. So I wrote my own tool.

Using the Tool

Simply run the executable. It requires the .NET Framework 4 Client profile, but otherwise needs no installation.

Select a running process from the dropdown list. Alternatively, type the name of a process (e.g. "Excel"), and wait for it to start; Mnemonic will automatically scan the process for as long as it runs.

You'll see a screen such as this:

Screenshot_chrome.png

Note that the 'scale' at the right-hand side of the graph shows 3071 MB - approximately 3GB. This is because this screenshot was running on a 32-bit Windows machine with the /3GB switch, which extends the user-mode virtual address space to nearly 3GB. On a standard 32-bit Windows environment, this limit would be 2GB, whereas under 64-bit Windows it would be 4GB (because Chrome is a Win32 process).

Note the large (approximately 1GB) yellow region to the right, which is reserved but not committed. You'll see this if you run a Win32 process under 32-bit Windows with /3GB, or under 64-bit Windows, unless your process is marked as 'large address aware'. If your process is marked with LARGEADDRESSAWARE, this region will be marked as Free (initially, at least); if not, Windows reserves it to prevent the process from accessing it.

Run your mouse over the graph to see details about the allocations, including address range and any module loaded (managed or unmanaged) at that range.

Control-click the graph to save it as a file. Specify the root name (and folder) for the files; a sequence number and the .png extension will be appended. Simply click the graph to save subsequent snapshots; the sequence number will increment.

Using the Code

The code consists of a reusable class for enumerating the contents of Virtual Memory: VirtualMemoryExplorer, and a simple front-end interface that allows the selection of a process and the graphical presentation of the information.

Two 'enumeration' loops take place within VirtualMemoryExplorer.Scan that build lists of VM regions that are of interest, and their characteristics.

The first looks at VM allocations:

C#
// Use UInt32 so that we can cope with addresses above 2GB in a /3GB 
// or "4GT" environment, or 64-bit Windows
UInt32 address = 0;
for (; ; address = (UInt32)m.BaseAddress + (UInt32)m.RegionSize)
{
    if (0 == VirtualQueryEx(processHandle, (UIntPtr)address, out m, 
                (uint)Marshal.SizeOf(m)))
    {
        // Record the 'end' of the address scale
        // (Expect 2GB in the case of a Win32 process running under 32-bit Windows, 
        // but may be extended to up to 3GB         
        // if the OS is configured for "4 GT tuning" with the /3GB switch
        // Expect 4GB in the case of a Win32 process running under 64-bit Windows)
        addressLimit = address;
        break;
    }
    VMChunkInfo chunk = new VMChunkInfo();
    chunk.regionAddress = (UInt32)m.BaseAddress;
    chunk.regionSize = (UInt32)m.RegionSize;
    chunk.type = (PageType)m.Type;
    chunk.state = (PageState)m.State;
    if ((chunk.type == PageType.Image) || (chunk.type == PageType.Mapped))
    {
        // .Net 4 maps assemblies into memory using the memory-mapped file mechanism;
        // they don't show up in Process.Modules list
        string fileName = GetMappedFileName(processHandle, chunk.regionAddress);
        if (fileName.Length > 0)
        {
            fileName = Path.GetFileName(fileName);
            chunk.regionName = fileName;
        }
    }
    chunkInfos.Add(chunk);
};

The second looks at the modules (DLLs) loaded by the process. I used the Process.Modules method. As per this page, from .NET 4, this list no longer includes managed assemblies - only unmanaged ones. (The only way to discover the managed assemblies is to use GetMappedFileName in conjunction with VirtualQueryEx, which is done in the snippet above).

C#
mappingInfos = new List<VMRegionInfo>();
foreach (ProcessModule module in process.Modules)
{
    VMRegionInfo mappingInfo = new VMRegionInfo();
    mappingInfo.regionAddress = (UInt32)module.BaseAddress;
    mappingInfo.regionSize = (UInt32)module.ModuleMemorySize;
    mappingInfo.regionName = Path.GetFileName(module.FileName);
    mappingInfos.Add(mappingInfo);
}
// Sort by address
mappingInfos.Sort(delegate(VMRegionInfo map1, VMRegionInfo map2)
{
    return Comparer<UInt32>.Default.Compare(map1.regionAddress, map2.regionAddress);
}); 

Points of Interest

I struggled with the process selection mechanism, which uses a DropDown. If the dropdown button is pressed, I wanted to show the list of running processes. If a name is typed into the textbox, I wanted Mnemonic to start scanning a process matching the typed name as soon as it starts. To achieve the behaviour I wanted, I eventually opted to scan for available processes in a background thread.

It was interesting to see that .NET applications are not, by default, marked as LARGEADDRESSAWARE and that mainstream applications such as the Office 2007 suite are not so marked, either. I'd be interested to know why this is.

I'm curious about the possibility of using memory-mapped files in managed applications (there is new support for this technology in .NET 4), so I was interested to see that the CLR uses memory-mapped files to access assemblies (DLLs) it loads into a process. These show up in purple in Mnemonic, and the assembly name is shown in the status bar.

History

  • 24 November 2011: First version
  • 9 February 2012: Updated the code to run under both 32- and 64-bit Windows, and to support both 32- and 64-bit processes

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Founder Pixel Analytics Ltd.
United Kingdom United Kingdom
My expertise is concentrated on image processing and analysis; I've been doing it most of my working life. Now I offer consultancy through my own company, as well as developing software products.

Comments and Discussions

 
GeneralMy vote of 5 Pin
SasiKumarC26-Jul-12 19:50
SasiKumarC26-Jul-12 19:50 
QuestionVMMap? Pin
Alois Kraus9-Feb-12 21:06
Alois Kraus9-Feb-12 21:06 
Altough you do not mention it I am sure you do know that VMMap exists?
It is simply the best tool out there for watching virtual memory, thread stacks, , file mappings, sharable pages, .... written by Mark Russinovich and Bryce Cogswell.

Good work because your tool is also interesting to show how to get the information via lesser known Windows APIs.

As for the 3GB switch. It is simply much more trouble than it is worth. Office and managed apps are no exception. There is a big number of libraries out there (including MS dlls) that is 32 bit and do crash when Large address aware is enabled. There is even a flag in the kernel to test this explictely by forcing the allocation from the top and not the bottom to force such errors immediately. The main problem was/is that if you use an int as storage for your pointers you will end up with negative numbers (>2GB) which does not work when you do pointer calcualations with integers and expect that all pointer values are positive.



Yours,
Alois Kraus

modified 10-Feb-12 3:24am.

AnswerRe: VMMap? Pin
Phil Atkin10-Feb-12 4:14
Phil Atkin10-Feb-12 4:14 
GeneralRe: VMMap? Pin
Alois Kraus10-Feb-12 4:35
Alois Kraus10-Feb-12 4:35 
GeneralRe: VMMap? Pin
Phil Atkin10-Feb-12 5:08
Phil Atkin10-Feb-12 5:08 
GeneralRe: VMMap? Pin
Alois Kraus10-Feb-12 6:08
Alois Kraus10-Feb-12 6:08 
GeneralRe: VMMap? Pin
Phil Atkin10-Feb-12 8:14
Phil Atkin10-Feb-12 8:14 
GeneralMy vote of 5 Pin
Kanasz Robert29-Nov-11 22:53
professionalKanasz Robert29-Nov-11 22:53 
GeneralRe: My vote of 5 Pin
Phil Atkin2-Dec-11 6:09
Phil Atkin2-Dec-11 6:09 
QuestionExport actual data Pin
peterchen24-Nov-11 23:44
peterchen24-Nov-11 23:44 
AnswerRe: Export actual data Pin
Phil Atkin25-Nov-11 4:59
Phil Atkin25-Nov-11 4:59 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.