|
We have a .NET console app that runs on our Win2003 server and processes a lot of network traffic.
After several days of running, we've discovered the app eating up all the memory on the machine -- nearly 2GB of memory. It brings the machine to a crawl, we start getting OutOfMemoryExceptions, all fun stuff.
Here's the kicker: I wrote some code so that I could type "garbage collect" in the console, and it would programmatically force a garbage collection (it calls GC.Collect()). I went to the console app this morning, see it throwing OutOfMemoryExceptions and eating 2GB of memory - I type "garbage collect" in the console, and the memory goes down to a reasonable size and the app magically starts working again, no OutOfMemoryExceptions.
Has anyone ever seen anything like this? Any ideas why the garbage collector isn't working?
|
|
|
|
|
Hi Judah,
I have no experience with Win2003 Server nor do I run programs that long. Anyway:
AFAIK there are only two ways to either run out or leak memory:
1. keeping references to managed objects that are no longer required;
2. not releasing unmanaged resources (i.e. forgetting to implement and/or call Dispose).
If your program runs out of memory, it must be violating 1 and/or 2. If it then recovers after
executing a particular piece of code, then it seems to me that piece of code is remedying 1, 2
or both. So I would suggest you inspect that piece to discover what is happening.
FWIW: If you want to implement some caching, make sure to use WeakReference objects.
BTW: 2GB in a couple of days isn't much, it is around 1MB/minute.
As an alternative idea, the gc probably uses some lower priority threads, so if your server is
heavily loaded, it just might fail to retrieve old generation objects in time. I suggest you
check its CPU load overall and in particular when the problems show.
Hope this helps.
|
|
|
|
|
Luc,
Thanks for the reply.
#1 doesn't seem right, since it wouldn't explain the garbage collection. If I was violating #1, for example, memory would still be high after I force a GC.
#2 doesn't seem right either, since forcing a garbage collection wouldn't magically free unmanaged resources, right? In other words, memory would still be high if I was forgetting to release unmanaged resources, since the unmanaged resources would never get freed, even if I force a GC.
What do you think?
|
|
|
|
|
Hi Judah,
if all it does is GC.Collect() then you are right of course, but since you did not show any
code my best guess was to assume something else also happens when you type "garbage collect"
on your console.
As for unmanaged resources, it is still not clear to me; my advice is to call Dispose() or Close()
[if it exists] as soon as on object is no longer needed; others claim a class with Dispose() also
has a finalizer that will take care of things, but I am not convinced of this.
As always: when the whole thing does not behave the way you expect it to, everything must be
suspected, al prior assumptions must be scrutinized.
|
|
|
|
|
Luc Pattyn wrote: if all it does is GC.Collect() then you are right of course
Yep, that's all it does.
Luc Pattyn wrote: As for unmanaged resources, it is still not clear to me; my advice is to call Dispose() or Close()
Yes, right. To the best of our knowledge, we're calling dispose everywhere possible. I guess I'm gonna have to do some profiling of the app over several weeks. Blech. Thanks anyways, Luc.
|
|
|
|
|
You might want to consider using PerfMon as well, and monitor the garbage collections.
|
|
|
|
|
You're welcome.
Good luck with the profiling, and don't hesitate to report your progress.
|
|
|
|
|
In MFC the Windows Message can be handled using ON_MESSAGE macro. How to do the same in C#?
|
|
|
|
|
I think you may want WndProc (or to override it anyway):
protected override void WndProc(Message m)
{
base.WndProc(ref m);
}
I could be wrong, i'm uncertain of what ON_MESSAGE is.
My current favourite word is: Bauble!
-SK Genius
|
|
|
|
|
Yes this what I wanted. I am surprised it's so simple. Thank you!
|
|
|
|
|
Hi
the below code is throwing up this error, Im stumped any ideas?thanks
string maxquery3 = "Select max(activity_order) from roomactivitylk where room_code = '" + room_code + "'";
//declare variable max
int max;
//sets max as the result from the sql command
SqlCommand cmd15 = new SqlCommand();
cmd15.Connection = con2;
cmd15.CommandText = maxquery3;
max = Convert.ToInt32(cmd15.ExecuteScalar());
//adds 1 to the value max
max = max;
++max;
|
|
|
|
|
My guess is the object in one of the fields is Null, and therefor cannot be made into an Int32?
Also, why the max = max; ++max; why not just max++; or ++max;?
My current favourite word is: Bauble!
-SK Genius
|
|
|
|
|
If the room code doesn't have anything associated with it, the select max would return null (technically it returns DBNull, but at the database end it's null). What you need to do is save the value to an intermediate variable and then convert it after testing for DBNull.Value.
object value = cmd15.ExecuteScalar();
if (value != DBNull.Value)
{
max = Convert.ToInt32(value);
}
max++;This is done off the top of my head so the syntax may be slightly wrong, but you should get the idea from it. More importantly though, don't use direct SQL like this. You leave yourself wide open to a SQL Injection Attack. Consider using parameterized queries instead.
|
|
|
|
|
Hi,
I'm trying to communicate with an unmanaged c++ dll from c# that takes two 2d
float arrays, one as input with data in it and the other will have the
result of the operation in it as follows:
bool Evaluate(float** input,float** output,int nFrames, int
featureWidth);
note: nFrames and featureWidth will determine the sizes of input and
output arrays in the c++ code.
Using this method in c# requires you to use float multidimensional
arrays. However I keep on getting exceptions thrown by the dll while
the same code works fine in a normal console application!
Trying to debug the dll as much as i can i figured out that the arrays
are not being passed correctly. can anybody help me out here, how
would I pass a 2 dimentional array to unmanaged c++ function?
Thanks
Fahd
|
|
|
|
|
Hi
how did you define the Call in c# (any MarshalAs usage or something..?)
what kind of exceptions do you get? access violations or others..?
f4hd wrote: dll while
the same code works fine in a normal console application!
do you mean you have a c# console application that successfully handles the dll?
greets
m@u
|
|
|
|
|
Hi M@u
thanks for the reply.
here is my definition
[DllImportAttribute(@".\xyz_MFC.dll")]
public static extern bool Evaluate(float [,] input, float[,] output, int nFrames,int featureWidth);
in unmanaged c++ it is
bool xyz_MFCApp::Evaluate(float** input, float **output,int nFrames,int featureWidth);
The exception that I get is System.NullReferenceException. Which is access violation I assume.
>>do you mean you have a c# console application that successfully handles the dll?
I have a legacy c++ code that im making into a dll in order to use it with c#
wen accessed using a c++ console app the code works fine. but wen i convert the code into a dll and use the exposed function in c# it throws the exception i just mentioned. on further investigating i figured out its the problem with the passing of 2d float arrays.
so if i pass the input array and access the first row i.e input[0][0] in the dll its fine but when i access input[1][0] the exception is thrown!
Thus i think i need to find out the correct way to pass the 2d array such that the memory it holds isnt affected.
hope I havent confused things further
Fahd
|
|
|
|
|
f4hd wrote: The exception that I get is System.NullReferenceException. Which is access violation I assume
No the Null Reference exception only says that you're passing a parameter that is null and is not supposed to be null
f4hd wrote: [DllImportAttribute(@".\xyz_MFC.dll")]
public static extern bool Evaluate(float [,] input, float[,] output, int nFrames,int featureWidth);
that with the multi-dimensional arrays might be a big problem.. i read something about it once and i think the central statement there was, that passing multi-dimensional arrays is not supported / only possible by using dirty tricks..
what you could try would be to mark the output array as ref or out.
[DllImportAttribute(@".\xyz_MFC.dll")]
public static extern bool Evaluate(float [,] input, [In,Out] ref float[,] output, int nFrames, int featureWidth);
or
[DllImportAttribute(@".\xyz_MFC.dll")]
public static extern bool Evaluate(float [,] input, [Out] out float[,] output, int nFrames, int featureWidth);
i find it a bit strange that nFrames is not a pointer, do you decide how many entrys per dimension the dll will return?
|
|
|
|
|
>No the Null Reference exception only says that you're passing a parameter that is null and is not >supposed to be null
well data is passed but incorrectly. see I can access the first row of the 2d array but not any other.
mebe you are right its not supported so one way, not quite elegant, is to convert the 2d arrays into 1d arrays during passing and receiving... Can there be any better way I wonder?
the In and out dont help much because of the same issue i mentioned above.
>>i find it a bit strange that nFrames is not a pointer, do you decide how many entrys per dimension >>the dll will return?
Yes I know it from before and is fixed for this Dll call and using it I do know the dimensions being returned
|
|
|
|
|
f4hd wrote: not quite elegant, is to convert the 2d arrays into 1d arrays during passing and receiving... Can there be any better way I wonder?
well if you can make it work that way i think that's a good way. an other way you could try would be to pass an array of intptrs that point each to an array of float.
it's ugly and i don't really know if it'll work at all but it *might* work
that it would look like this:
DllImportAttribute(@".\xyz_MFC.dll")]
public static extern bool Evaluate(IntPtr [] input, IntPtr[] output, int nFrames,int featureWidth);
private float[,] foo()
{
int nFrames = 3;
int featurWidth = 5;
float[,] myArray = new float[2,nFrames];
IntPtr[] input = writeArray(myArray);
IntPtr[] output = allocArray(featurWidth);
bool ok = Evaluate(input, output, nFrames, featurWidth);
freeArray(input);
return readArray(output,featurWidth);
}
private float[,] readArray(IntPtr[] Items, int featureWidth)
{
float[,] RetVal = new float[2,featureWidth];
readMemArray(Items[0],RetVal,0,featureWidth);
readMemArray(Items[1],RetVal,1,featureWidth)
freeArray(Items);
return RetVal;
}
private void readMemArray(IntPtr Pointer, Float[,] Target, int Dimension, int featureWidth)
{
float[] Dummy = new float[featureWidth];
Marshal.Copy(Pointer,Dummy,0,featureWidth);
copyArray(Dummy, Target, Dimension);
}
private void freeArray(IntPtr[] Items)
{
Marshal.FreeHGlobal(Items[0]);
Marshal.FreeHGlobal(Items[1]);
}
private IntPtr[] allocArray(int ItemCount)
{
IntPtr RetVal = new IntPtr[2];
int Count = Marshal.SizeOf(typeof(float)) * ItemCount;
RetVal[0] = Marshal.AllocHGlobal(Count);
RetVal[1] = Marshal.AllocHGlobal(Count);
return RetVal;
}
private IntPtr[] writeArray(float[,] Array)
{
IntPtr[] RetVal = allocArray(Array.GetLength(1));
float[] Dummy = new float[Array.GetLength(1)];
copyArray(Array, Dummy, 0);
Marshal.Copy(Dummy,0,RetVal[0],Dummy.Length);
copyArray(Array, Dummy, 1);
Marshal.Copy(Dummy,0,RetVal[1],Dummy.Length);
}
private void copyArray(float[] Source, float[,] Destination, int zDimIndex)
{
for(int i = 0; i< Source.Length; i++)
{
Destination[zDimIndex,i] = Source[i];
}
}
private void copyArray(float[,] Source, float[] Destination, int zDimIndex)
{
for(int i = 0; i < Destination.Length; i++)
{
Destination[i] = Source[zDimIndex,i];
}
}
the code is untested so maybe you'll have to change something but generally it could work.
however.. if you can make it work with an 1-dimensional array do it that way
greets
m@u
|
|
|
|
|
Thanks a lot for the code m@u
I havent tried it yet but I have tried the 2D to 1D method which worked as expected. I am worried about the speed impact of this method as it will be called many times and each time there is this conversion, its going to be slow. So if you can comment about the speed impact of the code you've mention in comparison to it that will be great.
Typical values of arrays will be something like 5000x40 and this dll will be called about a 1000 times! so if you think using Intptr might be quicker please let me know. Thanks
|
|
|
|
|
Hi
i don't really know because i didn't thest the posted code but i think the conversion will be quite slow for such big arrays and since you can make it work with the 2d to 1d conversion i'd say keep it like that.
greets
m@u
|
|
|
|
|
Ok, here are a few ideas for you to try:
1. Change the C# import declaration type from float[,] to float[][] (they are NOT equivalent types)
2. Change the C++ method signiture to take parameters of type float[][] instead of float**
3. Convert the float[][] to a float[] in your c# code, and change your method to either float* or float[] in your C++ code
4. Convert the float[][] to an IntPtr pointing to the address of the array
5. Convert the float[][] to a float* (requires unsafe code in C#)
6. Try combinations of the above, where you attempt a method signature in C++ using float[][], float[], float**, float*, and change the type you are sending from C# between IntPtr, float[], float*, float[][], float[,], float**, and in addition try mixing in the keywords out and ref as well
7. Do the same as you did in #6, but try copying the data to public memory using Marshal.Copy prior to making all the above calls
The above suggestions give you hundreds of combinations to try, and my guess is that at least one of them will work. Let us know if you get this figured out,
Sounds like somebody's got a case of the Mondays
-Jeff
|
|
|
|
|
Hi Jeff
Thanks for your answers.
1. Change the C# import declaration type from float[,] to float[][] (they are NOT equivalent types A:I know )
A: tried that before, gives a marshalling error.
2. Change the C++ method signiture to take parameters of type float[][] instead of float**
A: this doesnt work too, C++ gives lots of error if using flaot[][] instead of float**
3. Convert the float[][] to a float[] in your c# code, and change your method to either float* or float[] in your C++ code
A: I think this will work so if I got it right ill do the following:
float[,] --> float[] in c#
float* --> float** once in c++
float** --> float* when exiting from the function in c++
float[] --> float[][] in c#
it should work but isnt desireable.
4. Convert the float[][] to an IntPtr pointing to the address of the array
A:Can you help me with this please.
5. Convert the float[][] to a float* (requires unsafe code in C#)
A:Im not sure about this but perhaps dont want to try it right now.
6. Try combinations of the above, where you attempt a method signature in C++ using float[][], float[], float**, float*, and change the type you are sending from C# between IntPtr, float[], float*, float[][], float[,], float**, and in addition try mixing in the keywords out and ref as well
A have tried many combinations already including Out and Ref but in all ive noticed that because memory isnt properly handled i can only access part of the 2d array accessing other parts causes access violation.
7. Do the same as you did in #6, but try copying the data to public memory using Marshal.Copy prior to making all the above calls
A: I dont know about this. Need to dig in to it.
I did read about pinning pointers and stuff but all that is for C++ not C#, if Im not mistaken 'ref' takes care of it, but i may be wrong.
>>The above suggestions give you hundreds of combinations to try, and my guess is that at least one >>of them will work. Let us know if you get this figured out,
A: sure ill let u all know
kind regards
|
|
|
|
|
I want to know how the side by side versioning is maintain in dotnet.
1) if i used
<configuration>
<runtime>
<assemblybinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentassembly>
<assemblyidentity name="ClassLibraryversioning">
publicKeyToken="f690ee21283c8415"
culture="neutral"/>
<bindingredirect oldversion="1.0.0.0">
newVersion= "1.0.0.1"/>
</bindingredirect></assemblyidentity></dependentassembly>
</assemblybinding>
</runtime>
</configuration>
then it is always show same version(no compilation errors)
mean it always show the function in dll which i refered to the project.
even i specify bindingredirect
2) if i used following code then everthing is fine
{
Assembly objm = Assembly.Load("ClassLibraryversioning, Version=1.0.0.1,Culture=neutral, PublicKeyToken = f690ee21283c8415");
Type objmycom = objm.GetType("ClassLibraryversioning.Class1", true, true);
BindingFlags flags = (BindingFlags.NonPublic | BindingFlags.Public
| BindingFlags.Static | BindingFlags.Instance |
BindingFlags.DeclaredOnly);
MethodInfo m = objmycom.GetMethod("version", flags);
object myobj = Activator.CreateInstance(objmycom, false);
object result = m.Invoke(myobj, BindingFlags.InvokeMethod, null, null, CultureInfo.CurrentCulture);
MessageBox.Show(result.ToString());
}
tell me why it happened?
My problem is
|
|
|
|
|
Tell you what. Pick one of these posts and stick with it then delete the others by using the Delete link at the bottom of the message.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
