Click here to Skip to main content
15,888,610 members
Articles / Desktop Programming / ATL
Article

Software Verification's Memory Validator

23 May 2003CPOL 454.3K   37   69
Memory Validator is an automatic runtime error detection software tool for Windows applications.

This article is in the Product Showcase section for our sponsors at CodeProject. These articles are intended to provide you with information on products and services that we consider useful and of value to developers.

This is the first in a series of showcase reviews for our sponsors at CodeProject. These reviews are intended to provide you with information on products and services that we consider useful and of value to developers. They also allow you to discuss a product and interact with the manufacture or provider on a one-to-one basis.

Software Verification Logo

Faster leak detection, better analysis.

A common problem in software development is the problem of detecting and fixing memory leaks, handle leaks and memory corruption. Performing this task without the aid of software tools can be a slow and time consuming task, often resulting in a lot of wasted time as avenues of investigation turn out to be dead ends and the search for the bug turns to a different area of the software. This problem gets harder to solve and harder to manage the larger the codebase being worked on.

Software tools are often employed to solve this problem. Selection of an appropriate software tool requires that the software tool provides various features.

  • The software tool must not change the behaviour of the program being monitored.
  • The software tool must have as little impact as possible on the performance (both speed and reliability) of the program being monitored.
  • The software tool should provide many ways of interrogating the collected data as possible so that end users can look at a given problem from a number of angles.
  • The software tool should be configurable so that the user has a choice of tailoring the behaviour of the tool. This allows users to maximise the speed at which data collection occurs whilst collecting the data required, without collecting data that is not needed to investigate the bug under investigation.
  • If the program being monitored crashes, the software tool must not crash.
  • The software tool must be reliable, and in the event that the software tool crashes, it must not crash the program being monitored.
  • The software tool should also be capable of being used in a regression testing role, so that Quality Assurance departments can produce defect reports to direct software maintenance with a minimum of effort.

Memory Validator from Software Verification satisfies all of the criteria above.

Memory Validator is an automatic runtime error detection software tool for Windows applications. Memory Validator detects common problems associated with the use of dynamic memory allocation, Windows resources and handle management. Such problems include detecting memory leaks, resource leaks, handle leaks, incorrect allocations, buffer overruns, buffer under-runs, corrupted data, reference counting errors and un-initialised data. Memory Validator also provides support for detecting hotspots in the code where large amounts of memory or handles are allocated, reallocated or deallocated.

No special preparation of your program is required to use Memory Validator, simply use Memory Validator to launch your program and Memory Validator monitors your program's behaviour. You don't need to recompile or relink your program*. When your program performs an error, such as a double deallocation of memory or a buffer overwrite, Memory Validator tells you, showing you the error, the callstack for the error and the source code for the error. When you close your program, Memory Validator displays all the memory that has leaked. Each type of data displayed by Memory Validator is colour coded so that it is easy to identify. Powerful filtering mechanisms allow you to simplify the data that is collected and which data is displayed, so that you can concentrate on one particular error at a time.

expanded view of memory allocations

Memory Validator provides support for more than just the C runtime heap, you can also monitor the Win32 HeapAlloc(), LocalAlloc(), GlobalAlloc() and VirtualAlloc() functions as well as custom heap managers that you have written yourself. If you are using a reference counting scheme in your software, you will have come across the problem of out-of-sync reference counts and trying to determine where the counts go out of sync. Memory Validator provides support for tracking this type of error.

There are occasions when the program you want to monitor is already running, or the program you want to monitor is started by another program. Say for example your program is a COM server that starts when a document is embedded in a web-page. Memory Validator provides support for attaching to your server at the time the web-page requests the COM object. This can be very useful for identifying double frees or memory leaks that only occur when the program is run as a COM server, and not as a standalone program. If the program you want to monitor is already running, you can attach to the program with Memory Validator and monitor the program from this point to the program's conclusion.

Powerful analysis capabilities allow you to find memory allocations by address, near an address, allocations that hold pointers that point to an address (useful for determining which objects are still referencing the memory allocation you are examining) and which allocations are pointed to by pointers in the allocation you are examining (useful for determining which objects this object is using). You can find allocations by object type, or by the full or partial name of a function in the callstack of the allocations, or the full or partial name of the function at the top of the callstack in the name. You can find memory allocations by filename, dll name, address range, page range, size range, allocation id number (debug only), allocation heap and allocation type (alloc, realloc, free). On the analysis page, you can select an allocation and then ask to see all related objects. Related objects are objects of the same type, objects previously allocated at this address (good for analysing what may have happened after a delete that happened too early), other objects of the same type, other objects allocated in the same class, file or module, and so forth. Based on the results of an analysis you can select one, some or all of the results, and then repeat the query on these objects to drill through an entire allocation chain, seeing which object was related to which. This functionality can be very useful in finding the cause of very subtle bugs, which are often caused hundreds or thousands of allocations before the bug results in a visible error in the program.

Memory Validator provides a visual presentation of the current number of allocations for each type of memory and the total number of active handles, a visual presentation of the virtual memory map, with different colours for each type of memory use. This view can be very useful for determining why requests for large blocks of memory fail. Your computer may report X bytes free, but when you look at the memory map, you can see the memory is in 20 different blocks that are not contiguous.

Memory Validator provides support for detecting memory allocation hotspots. Once identified, the memory allocation and deallocation strategy can be modified to improve the way in which memory is consumed, thus putting less strain on the memory allocator, and so improving program performance.

Memory Validator provides support for ensuring that all places in the program that can allocated, reallocate or deallocate memory have been visited during test sessions. This support is provided by the Coverage window. The quality assurance team can use these metrics to ensure that an application has been thoroughly tested prior to launch.

Memory Validator provides support for regression testing. Regression testing can be performed in attended and unattended modes. The regression testing allows you to compare a baseline session with a session representing the test just performed. When the two sessions are compared you can see where you are making improvements in memory leak fixing and where you are making regressions in memory leak detection. You can export these results in HTML and XML formats for further processing. Unattended regression testing allows you to run Memory Validator from a command line or batch file, start the program of your choice, monitor for memory leaks, compare the resulting session with a baseline for the test, produce the regression test report, save the session and export the results in HTML and XML. For Quality Assurance departments with can use the regression testing facilities with their overnight test suites to provide at-a-glance reports the next day of which parts of the software are improving or regressing. Feedback from this can dramatically improve the error detection rate from the overnight/rolling software build. The XML reports can be further processed to produce reports which can be put on the internal bug report web server, or forwarded to management for further action.

Memory Validator is very configurable. The default options allow you to make immediate progress in tackling common problems. For more comprehensive error detection you can enable more options, or just the options related to your bugs. For example, if you are interested in tracking handle leaks, you don't want your software tool to track memory leaks - that would be a waste of time and effort. With Memory Validator, you can do just that, turn off the memory leak tracking and only track the handles. Or, say you wanted to track memory leaks, but only for the HeapAlloc(), HeapReAlloc() and HeapFree() functions, and not the C runtime. Turn off the CRT tracking, turn on the HeapAlloc() tracking and start your program. These are just two examples of improving the speed and ease at which you can find bugs with Memory Validator. The configurability goes further than this, you can, if so desired, go down to the function level to specify which functions should and should not be monitored.

Typical users of Memory Validator are software developers that don't want to spend a long time trying to identify what is causing the memory corruption, double delete, or memory leaks in their program. Quality Assurance departments will be interested in the support for unattended regression testing. Management will be pleased that data exported from Memory Validator can be turned in reports indicating improvements and regressions thanks to the support for XML export.

For professionals wanting to identify and fix hard to find memory leaks, resource leaks and memory corruptions in a fast and efficient manner, Memory Validator is a must have tool.

For sales and information on Memory Validator click here.

* To use the Memory Validator API, you need to link to a specific library file. This change will be required once only. You may need to change compiler settings to use the dynamic C runtime library if you are using the static C runtime library. This change will be required once only.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Web Developer
United Kingdom United Kingdom
Commercial Software developer.

Started out writing 6502 assembly language games for C64 (for a bet, I won, he never paid). I wrote 27 games in total.

After suffering and recovering Repetitive Strain Injury in 1994 I went independent and have consulted for BT, Aethos (GSM consultancy before sold to Logica), DERA, BT (again), Cloudworld (computer use for blind people), Solidworks (fastest growing company in history of CAD, ever).

I am currently working for Software Verification Limited, producing software tools for software engineers on the Win32 and .net platforms, targetting C++, C, C# and Java.


Comments and Discussions

 
GeneralRe: Good summary Pin
Stephen Kellett18-May-02 10:58
Stephen Kellett18-May-02 10:58 
GeneralGreat addition and a question Pin
Brent Corkum18-May-02 6:25
Brent Corkum18-May-02 6:25 
GeneralRe: Great addition and a question Pin
Stephen Kellett18-May-02 7:04
Stephen Kellett18-May-02 7:04 
GeneralRe: Great addition and a question Pin
Stephen Kellett18-May-02 7:20
Stephen Kellett18-May-02 7:20 
GeneralRe: Great addition and a question Pin
Brent Corkum20-May-02 12:12
Brent Corkum20-May-02 12:12 
GeneralRe: Great addition and a question Pin
Stephen Kellett20-May-02 13:58
Stephen Kellett20-May-02 13:58 
GeneralTell us about the RSI.... Pin
Christian Graus17-May-02 21:13
protectorChristian Graus17-May-02 21:13 
GeneralRe: Tell us about the RSI.... Pin
Stephen Kellett18-May-02 1:21
Stephen Kellett18-May-02 1:21 
Hi Christian,

RSI, a very complicated subject. I'll describe what happened to me. There is some strange bits of history that I'll include as for some readers, they may recognise bits of themselves.

I was first started showing signs of RSI in about 1988, I guess, which just manifest as back pain (early signs of a posture problem, not sitting properly, etc). I finally got ill at the local folk festival in July 1993. I noticed that the tingling I'd had in my hands in the past that had always gone away over a weekend, this time I had it all weekend and it was still there on Monday morning. Fast forward to December of the same year and I went horse riding (which always provided some release from the pain, because of the stretch tension provided by the reins), after that the release from the pain had gone in about 15 minutes. I hadn't even got to work. So I went in and told them I needed medical help. We managed to find an RSI specialist on Harley Street, London. I went to see him, practically passed out twice because the tests he did were so painful (and wouldn't be painful to a healthy person). A very expensive 20 minutes consultation. However he then found my a suitable specialist that he had trained that lived only 45 minutes from where I lived.

I got very ill, ill to the point where I couldn't hold a pint of beer (0.6 litres for you metric folk) with using both hands and being in pain. I ended being registered disabled and I banned myself from driving for a while (changing gear and turning the wheel were too painful). Later on, at one point, both of my arms were warm down to the wrists, and both my hands were cold and blue. Weird, but caused by RSI.

My RSI was caused by being a very prolific software engineer (I was 3 months *ahead* of the plans my manager had for the work I was doing at the time (X11/Motif on 7 Unix platforms, GIS work) - and yes his plans were realistic, everyone else was on schedule). Unreal. I was also playing guitar in the evening. This is referred to as "double hand use". In other words you never give your body any time to recover. The damage builds up over years and years. Often you get a small does of RSI and it goes away. My physio tells me that typically the third time you get it, it doesn't go away and thats the one that changes the rest of your life.

Fundamentally, RSI is a posture injury. Get the posture right and you'll start to heal. Get the posture right and you can do more, whilst damaging yourself less. At the end of the day your body is a machine, albeit an organic self repairing one (I sound like Babylon 5). If you use it incorrectly,through bad posture, or you overuse it, by over coding, or playing musical instruments or high impact sports (squash - high impact on your wrist), then you'll get damage.

There are many views on how to treat the many forms RSI. I was suffering from epicondylitis (tennis elbow), carpal tunnel syndrome (tired wrists is how you'll tell) and tendonitis. Fortunately I didn't get bursitis (spelling) or tenosynuvitis (which is where the fluid in your knuckles dries up, then your body overcompensates and produces so much your hands may end up looking like boxing gloves (no exaggeration).

I was treated by a phsio that specialised in back injuries and RSI. She stressed that a physio that is a generalist and not a specialist in RSI may do more damage because RSI is a specialist injury and treatment is not obvious. I was given a series of exercises, firstly to increase my range of movement (by the time I met her I had been off sick for a while, in bed and my posture was really bad). Once we had the range of movement back to normal, I was given more exercises to increase my strength (I had lost much of my strength as it was painful to do anything, so you tend to stop). This treatment lasted for about 6 months, when I could finally type again.

At that point I resigned my job (as I felt I was at the front of the redundancy queue because of my illness) and became an independent consultant. This was in part so that I could manage my RSI better by only working when I wasn't in pain. Some people think that RSI is an excuse for the lazy to claim sick pay. I find the view that RSI is a malingerers disease very offensive. Why would I want to be off work, when my mortgage and other bills are not being paid? So if anyone comes at you with that argument, point out to them that you are suffering from a genuine illness.

RSI may well cause you mental stress, as you don't know what to do, or how to cope, and don't know where to get advice. I had to spend months doing my exercises 3 or 4 times a day, 20 to 40 minutes each time. Then go for a long walk to improve my posture. I live in the middle of the fens (marsh flatlands) in East Anglia, so I had plenty of space to do this. However I couldn't handle what was happening at the time, and spent much of my time
in oblivion, courtesy of the local off-license.

Most of the exercises take the form of various stretches. Don't assume any old stretch will do. It won't. Get advice from a physio. And pay for it. Waiting on the national health, or whatever you have in your country, is wasting time. RSI doesn't wait for anyone. Also, I found that bypaying for the advice, I one day found myself not wanting to do the exercises. I was fed up. Then I thought "why am I paying this money out, only to ignore her advice?". That was a real turning point. The therapy they use on my was called
"Adverse Mechanical Tension", where they put stretches along your nerves to undo the nerve compression injuries that are common in RSI. To stretch a human nerve it needs to be stretched for 7 seconds (so count to 10). Nerves are elastic and don't stretch until they have been stretched for 7 seconds. Strange but true.

I still suffer from RSI, although I manage it a lot better. I no longer play the guitar, but do play mandolin, bagpipes and meleodeon (all of these need a shorter hand stretch to play). When my RSI gets worse, I stop playing the instruments, maybe do some research instead of programming work, go walking, and start doing the exercises again. I swim 4 times a week and take vitamin B6. Vitamin B6 is associated with tendon sheath healing. Other B vitamins are not a substitute for B6. In the UK, you need to go to your doctor to get sufficient does of B6 because then made this a restricted vitamin a few years ago because of media/health scare over-reaction.

Fast forward 8 years to the present day, and I'm working for Software Verification, working on the types of software tool I always found interesting.

RSI won't end your life, but it will change the way you live it.
If you treat it soon enough and learn to listen to your body (you'll get very good at this if you get ill enough) you can carry on pretty much as normal. I think one of the worst things for me, is that I often find holding hands can be quite painful, depending on my RSI. Thats not very nice when you are walking in a forest or on a beach with your girlfriend...

My advice to you, Christian, is to stop the out of work coding immediately (or after Monday so you can finish your screen saver Smile | :) , and see a suitably qualified physiotherapist. If you play any instruments or do any sports that are not running or swimming, again I'd stop doing those temporarily.

Get an egonomic keyboard, a split keyboard. Get a good quality chair. Set the chair up so that you sit up straight, with your hips slighlty higher than your knees. *Then*, and only then, setup the desk to suit the height you are now sitting at. That means propping the desk up, or shortening the desks legs.
Yes, the furniture should match your dimensions, not the other way around. Now
you've got the desk sorted out, set the monitor up, so that the top of the monitor screen is roughly level with your eyes. Measure an 11' angle and follow that. That should be where the bottom of the screen is. Roughly. Using that you can work out where the monitor should be (how far away).

Hand use:


  • From above, your forearm and hand should be in line (not bent).

  • From the side, your fore amd hand should be in line. Your wrist should not be below or above this line.


These requirements mean that you are putting less strain on your arms.

Don't let anyone inject you with cortisone. You can only have this 3 times in your life (it is not good for your bones). Also this is treating the symptoms, not the cause. Its a painful injection, I'm told, but does provide short term relief.

Carpal tunnel surgery. I'd advise against it, if you can get proper medical treatment you won't need it. I've heard some nightmare stories about it taking up to 6 months for the operation to heal. However I met a doctor last year, and she had had both her wrists done for this operation and was back at work a few weeks later. I guess there is a lot of variance with this operation.

Do seek advice. Don't think I'll live with it, it won't happen to me. Everyone thinks that and they always realise they were wrong, when it is too late.

I was lucky, not many people get as ill as I did and get to keep their career. I had no idea what I was going to do if I couldn't use my hands to type. This is just part of the story, but I have never gotten around to writing it.

Please seek medical advice

I hope that hasn't terrified anyone too much. The reality was worse than what I've written here.

I've tried to write down roughly most of what happened, even the darker bits. Its not completely in chronological order, as writing one things makes me think "I should mention this now.." and I change subject.

Do everything you can to prevent this happening to you.

Cheers

Stephen Kellett
--
Memory Validator. Faster Leak Detection, Better Analysis.
http://www.softwareverify.com

http://www.objmedia.demon.co.uk


GeneralMemory Validator V2.09 Pin
Stephen Kellett16-May-02 23:20
Stephen Kellett16-May-02 23:20 
GeneralRe: Memory Validator V2.09 Pin
17-May-02 19:46
suss17-May-02 19:46 
GeneralRe: Memory Validator V2.09 Pin
Stephen Kellett18-May-02 7:23
Stephen Kellett18-May-02 7:23 
Generalexcellent Pin
16-May-02 20:42
suss16-May-02 20:42 
GeneralRe: excellent Pin
Stephen Kellett16-May-02 23:13
Stephen Kellett16-May-02 23:13 
Generallooks like it has nice features, but.. Pin
16-May-02 3:28
suss16-May-02 3:28 
GeneralRe: looks like it has nice features, but.. Pin
Stephen Kellett16-May-02 4:36
Stephen Kellett16-May-02 4:36 
QuestionWhat is goin' on ? Pin
Armen Hakobyan15-May-02 21:53
professionalArmen Hakobyan15-May-02 21:53 
AnswerRe: What is goin' on ? Pin
Chris Losinger16-May-02 3:39
professionalChris Losinger16-May-02 3:39 
QuestionUnknown Section? Pin
James T. Johnson15-May-02 8:52
James T. Johnson15-May-02 8:52 
AnswerRe: Unknown Section? Pin
Chris Maunder15-May-02 16:14
cofounderChris Maunder15-May-02 16:14 
GeneralProduct Showcase == No Source Pin
Hans Dietrich15-May-02 6:10
mentorHans Dietrich15-May-02 6:10 
GeneralRe: CodeProject Pin
Maximilian Hänel15-May-02 5:35
Maximilian Hänel15-May-02 5:35 
GeneralRe: CodeProject Pin
15-May-02 7:25
suss15-May-02 7:25 
GeneralGreat Pin
15-May-02 4:02
suss15-May-02 4:02 
GeneralRe: Great Pin
15-May-02 8:51
suss15-May-02 8:51 
GeneralRe: Great Pin
Stephen Kellett16-May-02 1:11
Stephen Kellett16-May-02 1:11 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.