Re: Problems with a memory leak - C# Discussion Boards

7-Dec-02 11:10

I'm writing an app that will do things like display entries from the event log and a few other things. It cycles through 3 event logs, showing errors and warnings in a datagrid.

I was testing out the main app when I happened to have taskmanager (seen end note) open and saw that it was using 40+ meg for a program that was doing nothing really. This got me chasing down why it was using so much memory. So I reduce my program into a smaller set of features, with no add-on dll's and made a small test harness to log memory usage. This showed that on my win2k development server the program ran fine for an hour (<20 meg usage) but in the second hour it hit 40+ meg 3 times. I also ran it on my rc1 .net server and it show memory usage of 180+ meg in 30 mins. I then added in a check call the GC when the memory usage was over a fixed value (~16 meg) and logged whether this was called or not.

The logs showed that on my Win2k server the memory usage continued to climb even though the GC was called every tick of the timer. On my WInXP machine, the GC was called more frequently as time went on, but it stayed around the 16 meg limit.

Has anyone else seen something similar to this? It has frankly driven me up the wall so much I’ve put this project on hold while I write another app and hopefully find a fix for it.

Any one interested in helping or looking at the source, please contact me here or email me ( omega501@hotmail.com ) with a subject of "Request Source" so I don't delete it out of hand Smile | :)

Note: Taskmanager reports memory usage of running apps very badly. It doesn't show how much memory they are currently using, it shows how much it thinks the program will need ( I think). To verify this, open an app (word or similar, something big) and then open taskmanager, select the process tab and find the app you have just run. Select it (so it's easier to find) and then with task manager still open, minimize the app - I did this to MS Word and the memory usage went from 15 meg to 400k instantly - now I make all my .net apps check their own memory Smile | :)

Burt Harris7-Dec-02 15:17

7-Dec-02 15:17

In my experience, calling GC.Collect() is always a bad idea. It throws of the heuristics the garbage collector uses, and ends up costing you performance because it artificially advances objects into higher generations.

I'd suggest you apply the Allocation Profiler to find out if you've got managed memory problem. This tool is great.

Note that if you've got code using COM interop or P/Invoke, it possible that you've got a classic memory leak. Instead of using the task manager to monitor memory, you can use perfmon. Use the Process object's Virtual Memory, Private Bytes, and Working Set counters on your process, which show overall numbers, then add the ".NET CLR Memory" counter named "# Bytes in all heaps" for you process. If they both grow, your probably holding on to managed memory too long, if just the process ones grow, its unmanaged memory.

Burt Harris

Omega5017-Dec-02 16:27

7-Dec-02 16:27

Thanks. I'll check that out.

The reason I am calling GC explicitly was to see the impact on memory usage, ie. does it free up all the memory and from my logs it's not. In the logs (when graphed) you can easily see the high and low memory usage climbing over time - every time the GC kicks in it doesn't free up as much memory as the previous time. This is with or without calling GC explicitly. The only things I have in the program (outside of labels/textboxes) is the datagrid and a timer. The timer (2 secs) is used to call WMI to check cpu usage, and to read the event logs every 10 secs. Even without these added in, just running the timer to check memory usage and logging the results the memory still gets used in a similar, but smaller, cycle. I am checking the memory usage with this

<br />
    // get current instance of the app<br />
    CheckMem = Process.GetCurrentProcess();<br />
    //get current memory<br />
    CurrentMemory = CheckMem.PrivateMemorySize;<br />

Is there a problem with using this method?

To check the cycling behaviour I have avoided creating/destroying objects as much as possible, and I am now just changing the values of the objects
(ie. clearing the datatable that is the source for the datagrid, and programatically adding in new rows as relevent, both the datatable and the datarow are defined "Globally" (to use vb terms, not sure of the c# term)).

I am either explicitly destroying objects when they are finished, or implicitly when the function has finished its job. Where I know I will be using them elsewhere I make them global. This doesn't seem to have stopped the cycling, but it does seem to have slowed it down.

Burt Harris8-Dec-02 11:42

8-Dec-02 11:42

With a call to Process.GetCurrentProcess() inside a timer event routine, I saw working set growth similar to what you described. The trick is every time you call Process.GetCurrentProcess, it's creating a new object, and it's a fairly heavyweight object...

Try moving the call to Process.GetCurrentProcess() outside of your timer event routine. I made it a member of my Form class and memory growth went away completely. Since its not going to change, you could even make it a static member.

static private Process p = Process.GetCurrentProcess()

private void timer1_Elapsed(object sender, System.Timers.ElapsedEventArgs e) {
    label3.Text = DateTime.Now.ToString("hh:mm:ss");
    label1.Text = p.PrivateMemorySize.ToString();
    label4.Text = p.WorkingSet.ToString();
}

Creating the Process object just once is the real fix for you, but the other thing to point out is that this object implements IDisposable. Whenever an object implements IDisposable, you'll get much better perormance if you manage it's lifetime, and call Dispose() on it when done. This is so important, the C# language added the using statement just for this.

Ignoring the advise of my second paragraph, I left the Process.GetCurrentProcess inside the timer event routine, but wrote it like this:

using (Process p = Process.GetCurrentProcess()) {
    label1.Text = p.PrivateMemorySize.ToString();
    label4.Text = p.WorkingSet.ToString();
}

Net result: Memory grew a little, but stabilized.

So here's what all this means. Creating a System.Diagnostics.Process directly or indirectly allocates a chunk of unmanaged memory. This unmanaged memory isn't released until either: 1) IDisposable.Dispose is called (implied in my using statement), or 2) the object is garbage collected.

Option 1 (Dispose) is far preferred to option 2, because 2 is far less predictable. But for this paricular case, option 0 (creating the object once and reusing it) is the best.

Burt Harris

Omega5019-Dec-02 10:52

9-Dec-02 10:52

I tried out those suggestions but there still seems to be a problem.

Trying the

<br />
using (Process p = Process.GetCurrentProcess){<br />
...<br />
}<br />

method, I was still showing a similar memory usage. When I tried the

static private Process p = Process.GetCurrentProcess()

method, I was amazed - No memory usage increase was found once the program started. I ran the program overnight and the memory values where the same in the morning as when the program started. Is this saying there isn't a memory leak or that it isn't updating the values properly?

This was using

<br />
static private Process p = Process.GetCurrentProcess();<br />
...<br />
<br />
private void GetCurrentMemory(){<br />
  intCurrentMem = p.WorkingSet;<br />
  // or<br />
  intCurrentMem = p.PrivateMemorySize;<br />
<br />

Both of these returned non-changing values. Am I right in thinking that over a period of 10 hours, there should be some change in memory usage for a program? This program goes out and rotates through 3 event logs (every 10 secs for testing purposes) looking for errors and when an error is found it checks to see if it has already recorded that error in the datatable (I don't want to see 10 entries of "cd-rom has a bad block" or similar, once is enough). This is a fair amount of work that it's doing but using the static process object the memory doesn't change. Changing it to the "Using(...)" format brings back the memory creep though.

Burt Harris9-Dec-02 21:28

9-Dec-02 21:28

Yup, your right, the static approach is misleading. You can fix it by calling p.Refresh(). The memory isn't exactly static in my simple example, but it stabilizes after a few minutes. I saw the same basic pattern with the using statement.

None the less, the bytes/second allocated seems quite high. I ran the allocaiton profiler on it, and found that the bulk of memory dynamically allocated for my simple test was in System.Diagnostics.ThreadInfo. Again, it looks like the System.Diagnostics.Process class is rather heavyweight for your purpose. I guess it's snapshotting a whole bunch of information you don't need...

You can get working set size from the Environment object, it's lots cheaper, but even with this, I do see an ongoing increase in private bytes by just reading Envrionment.WorkingSet. Interesting... Allocation Profiler says its related to security checks code, that's more what I expect. I leave it running for quite some time and eventually the GC kicks in and trims working set back. I haven't left this running overnight, but I will...

Burt Harris

Omega50111-Dec-02 22:14

11-Dec-02 22:14

I added in the .Refresh() and the memory usage started changing like it used to. Seeing this, I left it running for a few hours and after 7+ hours, the memory usage has a peak of 110 meg (set 5 mins ago), a current memory of 60 meg, and it never dropped below its initial value of 20 meg since the app started.

The main memory usage seems to occur when reading the event log, as when it is checking cpu usage the memory changes a small amount (~100k) but seems stable. It seems to be having problems when reading the event log into a datatable and then displaying that in a datagrid.

It also seems, since I made the Process object static, that a) initial memory usage has jumped (from 10 meg to 20 meg), and that peak memory usage has doubled as well (previously it peaked around 40 - 50 meg, now it's 110 meg).

I am not talking about a lot of data being stored in memory consistently - the datatable has a max of 10 records, and I clear the datatable at the beggining of each loop through the current log (cycling through system, security, application logs every 10 secs for testing purposes).

Is it the Process that is using up so much memory? - I call the p.WorkingSet every 2 secs currently, but now I am only calling the
Refresh(), not creating a new instance of the object. Is there a more "light weight" way of checking the memory usage of a the current app? Maybe calling the Process object is itself causing the high memory usage?

David Stone8-Dec-02 5:56

David Stone

8-Dec-02 5:56

Welcome to CP Burt! It's always nice to have more people from Microsoft here to answer questions. Wink | ;)

So you work in the Speech group huh? Gotta say, I love the .NET Speech SDK. Very nice...even more so that it works with ASP.NET. Cool | :cool:

I send my kudos along to your team.

How did you find out about CP? Was it purely accidental, a coworker mentioning it to you, or was there an internal memo that everybody should monitor what Nick says in his column? Big Grin | :-D

I don't know whether it's just the light but I swear the database server gives me dirty looks everytime I wander past.
-Chris Maunder

Microsoft has reinvented the wheel, this time they made it round.
-Peterchen on VS.NET

Burt Harris8-Dec-02 10:50

8-Dec-02 10:50

Thank you David. Glad you like the .NET Speech SDK. I can't take much credit for it, I've been working on a related project that hasn't shipped yet, but if you like the SDK, we're probably on the right track.

I found CodeProject tracking down rumors that turned out to be Pavel Zolnikov's Command Prompt Explorer Bar. Now that's some cool software. I'm still studying it, but already I'm impressed.

So there wasn't any sort of all-hands memo, but I will say that there is awareness inside Microsoft as to how important it is we contribute to the developer community. I heard a VP up my reporting chain say he tries to start and end each day solving a customer problem, on message boards or elsewhere. I don't know what forums he haunts, but that's pretty cool.

Topics like this seem like a good place for me. I've got solid server-grade code written in C#, and handling audio data with near real-time requirements. There are a lot of people, (even within Microsoft), who are skeptical of managed code, but my team has taken the plunge, and making it work for our somewhat challenging performance goals hasn't been that hard.

Turning back to the original topic of this thread, what I've learned is that managed memory doesn't mean you won't have memory problems: there are new kinds of memory problems. Understanding the memory allocation behavior your apps is key. With a tools like the allocation profiler, and a little patience, I think most developers will find that solving the new style memory problems (generally performance) a lot easier to diagnose and address than old-style ones (like dangling pointers).

Even with tuning, managed apps will generally use more memory than unmanaged ones, but very high memory consumption indicates a bug in either.

Burt Harris

David Stone8-Dec-02 17:09

David Stone

8-Dec-02 17:09

Burt Harris (msft) wrote:
I found CodeProject tracking down rumors that turned out to be Pavel Zolnikov's Command Prompt Explorer Bar. Now that's some cool software. I'm still studying it, but already I'm impressed.

Laugh | :laugh:

I think everybody loves that thing! I use it all the time. It's worth the extra few seconds it takes to start up cmd.exe and put it in the window.

Burt Harris (msft) wrote:
So there wasn't any sort of all-hands memo, but I will say that there is awareness inside Microsoft as to how important it is we contribute to the developer community. I heard a VP up my reporting chain say he tries to start and end each day solving a customer problem, on message boards or elsewhere. I don't know what forums he haunts, but that's pretty cool.

That's really cool. I'm glad to know that most of you guys really are concerned with our comments. I for one, laugh at people who say that Microsoft is a huge corporate giant that doesn't care. And I can do that because of people like you, your VP, Eric Gunnerson, and Nick Hodapp. I must say kudos to you guys. It's a lot easier to solve a problem when someone with the inside scoop comes along to help you out.

EndDialog

peterchen7-Dec-02 10:02

peterchen

7-Dec-02 10:02

My quest for .NET equivalents doesn't seem to end.

How do I "EndDialog" a Form? Close() works, but I can't pass a result for ShowDialog.

I already found the Button.DialogResult member, but I want to do some extra processing before closing the dialog.

If I could find a souvenir / just to prove the world was here [sighist]

Re: EndDialog

Burt Harris7-Dec-02 10:13

7-Dec-02 10:13

Funny, I thought I replied to this already.

You can override the value assigned to the DialogResult property when the user clicks the Close button by setting the DialogResult property in an event handler for the Closing event of the form.

Burt Harris

Re: EndDialog

peterchen7-Dec-02 10:47

peterchen

7-Dec-02 10:47

Burt Harris (msft) wrote:
Funny, I thought I replied to this already.

yep, got three notifs Cool | :cool:

thanks!

If I could find a souvenir / just to prove the world was here [sighist]

plz ans these

imran_rafique7-Dec-02 9:23

imran_rafique

7-Dec-02 9:23

Q1-if a system is in loggoff state and i call
GetUserName(..).
is the function fails,and return value is zero?
and how did i know that the system is in loggoff state?
//////////////////////////////////////////////////////////

Q2-how i will trap the WlxDialogbox and where msgina asks for the userid,password
and fill in the fields by my self?
can any body help me in this regard?
//////////////////////////////////////////////////////////

Q3-how did i find the system32 path programatically?
//////////////////////////////////////////////////////////

Q4-if i call BlockInput(true)
and then i want to use postmessage or sendmessages for keyboard and mouse
will these messages works?
/////////////////////////////////////////////////////

r00d0034@yahoo.com

Re: plz ans these

Burt Harris7-Dec-02 10:02

Re: Custom TextBox Drawing

7-Dec-02 10:02

Sounds like your trying to replace the login dialog. This is intentionally difficult (near impossible) to do. NT/W2K/XP guarentees that when you type Ctrl+Alt+Delete, the dialog is the trusted login dialog, and not something that might steal your password.

If you want auto-login, there is a registry setting that lets you specify a username and password. I'd only reccomend using this if the machine is in a physcially secure location.

Burt Harris

Custom TextBox Drawing

mikasa7-Dec-02 8:13

mikasa

7-Dec-02 8:13

I am trying to do some Custom Drawing within the TextBox in .NET. I have finally figured out how to do it and the gradient Background / ForeColors look great! However, there is a big drawback. It seems that when the TextBox receives focus, for some reason it makes the Text twice the size (from 8pt to 16pt)! WTF | :WTF:

Would anyone know why this is happening? I can't really post the Source Code because there is a lot! I am Deriving from the TextBox and Setting the Styles on it to:

SetStyle(ControlStyles.UserPaint, True)
SetStyle(ControlStyles.ResizeRedraw, True)
SetStyle(ControlStyles.DoubleBuffer, True)
SetStyle(ControlStyles.AllPaintingInWmPaint, True)

The TextBox then handles mainly the OnPaint & OnPaintBackground. I've also had to handle the OnLostFocus / OnGotFocus to do painting to override the Base painting since Painting events do not fire unless all the Text is Selected.

Burt Harris7-Dec-02 10:05