Advanced Debugging Tips

Question

0.00/5 (No votes)

See more:

I have been facing some issues lately with Pinvoke calls to native functions of advapi32.dll from managed code and i haven't found yet a way to make it work properly, facing multiple problems related to the classic "A call to ... has unbalanced the stack" and "attempt to read/write protected memory".

My question here is regarding on how to debug those situations properly, as i may not have experience debugging this i ran into the situation where i cannot get information enough to know where the error is.

Is there any way to debug/step into those calls from VS2010 that im not aware of, or maybe a way to read the memory that is being allocated and try to discover it by myself?

To give an example, i have a PTR to a memory address which is supposed to hold a structure, which is pretty big and consist of a lot of another sub-structures and i believe the problem is related to the definition of those as i might not be using the correct data types.
So, for instance, how i can read a specific memory block from a PTR and see which data is allocated there to see if it fits into the structure in the correct fields?

I'm sorry if this is a confusion question, but I'm more like looking for some guidelines and tips into this... hope to get some comments.

Posted 8-Feb-12 12:14pm

creizlein

Add a Solution

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Dave Kreskowiak · Answer 1 · 2012-02-08T12:28:00

Solution 1

The biggest culprit of "A call to ... has unbalanced the stack" is you declared a type as Long instead of Integer. This is very common if you used a Declare you found on the internet and it belongs to VB6 code.

A Long in VB6 is a 32-bit signed integer. A Long in VB.NET is a 64-bit signed integer. Since the field sizes are not different than the 32-bit field the API expected, you just unbalanced the stack.

For example, you're trying to call an API function that expects a signed 32-bit integer, so (in VB6) you Declare that parameter a Long. Your VB.NET code, using the same Declare, just wrote a 64-bit signed integer (8 bytes) to the stack. Now, the API function, expecting a 32-bit number, pops 4 bytes off the stack. When the call returns, VB.NET expects the stack pointer to point to a certain location, and its now pointing 4 bytes away. Uh Oh!

Posted 8-Feb-12 12:28pm

Dave Kreskowiak

Comments

creizlein 8-Feb-12 22:05pm

Yeah, im pretty much aware of that, thanks for the reminder anyway, but the strctures im handling do declare like 50 (counting nested structures) integers/longs and its turning complicate to understand which should be 4bytes and which should be 8, thats is why im looking into a way to read the memory...

Dave Kreskowiak 8-Feb-12 23:33pm

Dare I ask which structures?? I'm not aware of anything that goes THAT deep, and I've P/Invoked the code signing API's (NOT FUN!!)

creizlein 9-Feb-12 8:47am

Sure, the structures in question are EVENT_TRACE, EVENT_TRACE_HEADER, ETW_BUFFER_CONTEXT, TRACE_LOGFILE_HEADER and EVENT_TRACE_LOGFILE. its all regarding native ETW.

Dave Kreskowiak 9-Feb-12 17:32pm

Are you writing events to ETW or listening for them??

creizlein 9-Feb-12 17:54pm

I'm reading events in realtime mode, from the Kernel Logger
So far i was able to figure out the problem, for some reason the problem does not belongs to the structures itself, but on the Event/Buffer Callbacks.
According to my tests, in Vista x86 and Win7 x86 the callbacks need to be declared with a parameter by reference lets say void EventProcess(ref event_record e);
But in Win7 x64 which is my development machine the code will fail and it expect the parameter to be just a input value, and not by reference, which makes more sense to me, and i have though of changing it.

But it still making random crash when more memory from the program is in use, looks like the GC is causing some troubles as well, but everything runs fine on x64 so its really driving me crazy ;(