Adplus: Handling Managed Exceptions

• Download source code - 3.5 KB

In this post, I’m going to concentrate on managed exceptions and the ways in which we can handle them using Adplus. Some time ago, John Robbins in his Bugslayer column wrote a great article about handling specific managed exceptions using Adplus. Although the general concept of the managed exception hasn’t changed (and so the SOS commands), his scripts do not work with the latest installation of Adplus. With the version 6.12 of the Debugging Tools for Windows, we received a brand new version of Adplus (7^th) which was transformed from a VB script to a .NET application. Although the command line options seem to be compatible with the previous version, the configuration file structure changed significantly. Old scripts need to be rewritten in order to work with the new version. In this post, I will not only upgrade John Robbin’s scripts but also show you some (I hope interesting) new features of Adplus.

Creating Adplus Configuration File

As command line parameters do not allow us to use many Adplus advanced features, I generally prefer to store Adplus run settings in config files with names explaining their usage. In this post, we will work on some sections of the configuration file but I strongly encourage you to read the Adplus documentation and find detailed information about the configuration file structure. We will start from the following log-exceptions.adplus.config file (all samples can be downloaded from the blog samples website):

<adplus version="2">
  <settings>
    <runmode>crash</runmode>
    <lastscriptcommand>qd</lastscriptcommand>
  </settings>
  <exceptions>
    <allexceptions>
      <actions1>LOG</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>LOG</actions2>
      <returnaction2>GN</returnaction2>
    </allexceptions>
    <allevents>
      <actions1>LOG</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>LOG</actions2>
      <returnaction2>GN</returnaction2>
    </allevents>
  </exceptions>
</adplus>

If we use the above configuration file with the Thrower.exe application (can be downloaded from here):

> adplus -c log-exceptions.adplus.config -o c:\dumps -sc Thrower.exe

Adplus will geneate three files in the output directory:

• ADPlust_report.txt – contains the settings with which the Adplus was run
• DebuggerScript.txt – the script generated by the Adplus and executed in the debugger at the beginning of the debugging process
• Adplus_log – all the ouput produced by the debugger

If we look at the Adplus_log, we can see that not much information was provided about the exceptions that occurred:

...
(12d8.1a14): CLR exception - code e0434352 (first chance)
FirstChance_clr_NET_CLR
ModLoad: 000007fe`e5e30000 000007fe`e5f1a000   
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\diasymreader.dll
FirstChance_ld_DLL_Load
(12d8.1a14): CLR exception - code e0434352 (first chance)
FirstChance_clr_NET_CLR
(12d8.1a14): CLR exception - code e0434352 (first chance)
FirstChance_clr_NET_CLR
ModLoad: 000007fe`fca50000 000007fe`fca5c000   C:\Windows\system32\VERSION.dll
FirstChance_ld_DLL_Load
(12d8.1a14): CLR exception - code e0434352 (!!! second chance !!!)
SecondChance_clr_NET_CLR
...

We can only deduce that there were three CLR exceptions (code e0434352) from which two were handled in the application code (only first chance logged by the debugger) and the third broke the application (thus second chance logged). Unfortunately, we can’t either see what the exception type was or who threw it. As cdb (internally used by Adplus) is a native debugger, it does not understand .NET metadata concept. In order to work with .NET types and methods, we need to use a native-managed translator, example of which is a Windbg SOS extension. It’s by default placed in the .NET Framework installation directory and we just need to instruct Adplus how to load it. For this purpose, we can modify the postcommands section of the configuration file. In Adplus v7, all actions must be defined as keywords and only in keywords we can issue the debugger commands. Our modified log-exceptions-and-load-sos.adplus.config file will look as follows:

<adplus version="2">
  <keywords>
    <!-- 
    If you are attaching to the process just use: .loadby sos clr.
    
    If it's .net 2.0 you need to use: sxe -c ".loadby sos mscorwks;gn" ld:mscorjit
    or only .loadby sos mscorwks if you are attaching to the process
     -->
    <keyword name="LoadSOS">sxe -c ".loadby sos clr;gn" ld:clrjit</keyword>
  </keywords>
  <settings>
    <runmode>crash</runmode>
    <lastscriptcommand>qd</lastscriptcommand>
  </settings>
  <postcommands>
    <debugactions>LoadSOS</debugactions>
  </postcommands>
  <exceptions>
    <allexceptions>
      <actions1>LOG</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>LOG</actions2>
      <returnaction2>GN</returnaction2>
    </allexceptions>
    <allevents>
      <actions1>LOG</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>LOG</actions2>
      <returnaction2>GN</returnaction2>
    </allevents>
  </exceptions>
</adplus>

Logging the Managed Exception Type and Stack Trace

At this point, we should have a SOS extension loaded into the debugger and thus be able to use commands that it provides. Let’s define two new keywords (help on the SOS commands can be found on MSDN):

...
<keyword name="PrintManagedException">!PrintException</keyword>
<keyword name="ManagedStack">!CLRStack -a</keyword>
...

Next step is to define new exception type handling in the exceptions section of the configuration file. As I’ve already pointed, the cdb debugger is not aware of .NET exception types. Fortunately, all framework exceptions have the same code and so it’s easy to write a general handler for them. Our final log-managed-exceptions.adplus.config will be as follows (notice that we are concerned only about 1^st chance exception):

<adplus version="2">
  <keywords>
    <!-- 
    If you are attaching to the process just use: .loadby sos clr.
    
    If it's .net 2.0 you need to use: sxe -c ".loadby sos mscorwks;gn" ld:mscorjit
    or only .loadby sos mscorwks if you are attaching to the process
     -->
    <keyword name="LoadSOS">sxe -c ".loadby sos clr;gn" ld:clrjit</keyword>
    <keyword name="PrintManagedException">!PrintException</keyword>
    <keyword name="ManagedStack">!CLRStack -a</keyword>
  </keywords>
  <settings>
    <runmode>crash</runmode>
    <lastscriptcommand>qd</lastscriptcommand>
  </settings>
  <postcommands>
    <debugactions>LoadSOS</debugactions>
  </postcommands>
  <exceptions>
    <allexceptions>
      <actions1>LOG</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>LOG</actions2>
      <returnaction2>GN</returnaction2>
    </allexceptions>
    <allevents>
      <actions1>LOG</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>LOG</actions2>
      <returnaction2>GN</returnaction2>
    </allevents>
    <exception code="clr">
      <actions1>PrintManagedException;ManagedStack</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>Log</actions2>
      <returnaction2>GN</returnaction2>
    </exception>
  </exceptions>
</adplus>

After issuing:

> adplus -c log-managed-exceptions.adplus.config -o c:\dumps -sc Thrower.exe

in the Adplus_log, we should have entries similar to this one:

(ef8.145c): CLR exception - code e0434352 (first chance)
Exception object: 00000000024e1778
Exception type:   System.InvalidOperationException
Message:          ex1
InnerException:   <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131509
OS Thread Id: 0x145c (0)
Child SP         IP               Call Site
000000000013ee38 000007fefda8cacd [HelperMethodFrame: 000000000013ee38] 
000000000013ef80 000007ff0016019f Thrower.Main(System.String[])
    PARAMETERS:
        args (0x000000000013f020) = 0x00000000024e16f0
    LOCALS:
        0x000000000013efa8 = 0x0000000000000000
        0x000000000013efb0 = 0x0000000000000000

which gives us much more information about what happened in the application.

Creating a Dump When a Managed Exception Occurs

Very often, the above Adplus_log is not enough to successfully diagnose the exception’s root cause. At these moments, you might try to add just few more debugger commands to keywords section (for locks, processes or threads), but definitely the best solution would be to have a memory dump (snapshot) taken at the time of the failure. With a full memory dump, you will be able to examine the stacks of all the threads, read garbage collector heap(s) or find lock acquisitions. Native debuggers (such as cdb or windbg) allow you to control what should be written to the dump file. Read help of a .dump command to see all the options available. However, in managed debugging scenarios, most of the times, you will need a full memory dump (created with the \ma switch). There is already an Adplus built-in keyword for creating this kind of dumps and its name is (I bet you guessed ) FullDump. Let’s try it in our CLR first chance exceptions:

...
  <exceptions>
    <exception code="clr">
      <actions1>PrintManagedException;ManagedStack;FullDump</actions1>
      <returnaction1>GN</returnaction1>
      <actions2>Log;FullDump</actions2>
      <returnaction2>GN</returnaction2>
    </exception>
  </exceptions>
...

After running the Thrower application under Adplus, you should have noticed that four dump files were generated in the output directory. Unfortunately, the time needed to write dumps to the disk impacts the application performance and responsiveness, especially when the number of thrown exceptions is high (though this often signifies a buggy application) or when the application consumes big chunks of memory. Sometimes, your process runs in a strict environment and if it does not respond in a predefined amount of time, it will be killed (IIS application pool might be a good example here). For reasons listed above and all others that might come to your head, you should generate memory dumps carefully and only when it’s really necessary. It might be a good general practice to first run application using log-managed-exceptions.adplus.config file (eventually generating full dumps only for the failures – second chance exceptions). After analyzing the generated logs, you should find exceptions that raise your suspicion and generate full dumps only for them. This is the subject of the next paragraph.

Filtering Exceptions

So you’ve found exception types for which you would like Adplus to generate full memory dumps. There is a special SOS command !StopOnException that will help us here. When issued in the debugger, it adds a new handling mechanism for the clr exception:

0:000> !soe -create System.ArgumentNullException 1
Breakpoint set
0:000> sx
...
 clr - CLR exception - break - not handled
       Command: "!soe  System.ArgumentNullException 1;
       .if(@$t1==0) {g} .else {.echo 'System.ArgumentNullException hit'}"
...

Moving back to Adplus, let’s define two new keywords in the keywords section:

<keywords>
...
    <keyword name="FullDumpOnSystemArgumentException">!soe  System.ArgumentNullException 1;
    .if(@$t1==1) {.dump /ma /u ${AdpDumpDir}\FULLDUMP_SystemArgumentException_${AdpProcName}_.dmp}
    </keyword>
    <keyword name="FullDumpOnSystemInvalidOperationException">!soe  System.InvalidOperationException 1;
    .if(@$t1==1) {.dump /ma /u ${AdpDumpDir}\FULLDUMP_SystemInvalidOperationException_$
    {AdpProcName}_.dmp}</keyword>
</keywords>
...

Now, we can modify the exception handling logic and use our newly defined keywords (file log-and-dump-specific-managed-exceptions.adplus.config from samples):

...
  <exceptions>
    <exception code="clr">
      <actions1>
         PrintManagedException;
         ManagedStack;
         FullDumpOnSystemArgumentException;
         FullDumpOnSystemInvalidOperationException
      </actions1>
      <returnaction1>GN</returnaction1>
      <actions2>Log;FullDump</actions2>
      <returnaction2>GN</returnaction2>
    </exception>
  </exceptions>
...

After running Adplus with the new configuration file, it should generate two full dumps for the first chance System.ArgumentNullException and System.InvalidOperationException and one full dump for the second chance System.NotImplementedException.

I hope that you find this article helpful – if you would like to play with Adplus, all configuration files as well as the Thrower application can be downloaded from here. Happy debugging!