am a big fan of using right tool for generating process dumps depending upon
what type of scenario you are debugging and what type of constraints you are
facing. I have blogged about the case for ADPlus,
DebugDiag & ProcDump . These are not the only tools available for creating dump files and there are
other means of generating process dumps as well. In general these tools provide
enough flexibility and options for generating dump file for widely needed problem
categories like Application Crashes, hangs, memory issues, slow performance
etc. However sometimes there are situations where you need more flexibility on
what event should trigger dump file generation. This article describes how this
could be done using Performance Counters, Windows Task Scheduler and ADPlus.
article assumes you are familiar with basics of production debugging and have
some basic know how of dump files. If not, you can watch my video on this topic
Performance Counters provide useful information about how
well operation system or any given application is performing. This information
could be used to monitor overall health and reliability of application and also
to troubleshoot performance bottlenecks etc. You can also create custom counters and use these to keep track of your application health and stability.
In this article, I will demonstrate how you can automate generation of dump
files based on threshold defined on these custom performance counters.
Creating Custom Counters
There are several ways of creating customer counters. Covering
all possible methods of creating these custom counters is out of scope for this
article, however, I will only discuss steps involved in creating a custom
counter using Server Explorer. If you need to learn more about creating custom
counters, the article found here
could be a good start.
Using Server Explorer is probably simplest way of creating
custom performance counter. Just launch Server Explorer from using Visual
Studio, right click on Performance Counter node and select Create New Category option as shown in Figure 1.
Figure 1: Performance Counter context menu
You will get Performance Counter Builder dialog where you can enter information about Category and also to add individual Performance counters withint this category as shown in figure 2.
Figure 2: Performance Counter Builder Dialog
Let's assume that we have start using and publishing this counter in application. Also assume that once in a while TransactionsCount is getting quite high in a production environment and we are not been able to re-produce this problem in house. This situation obviously will make debugging and troubleshooting this problem quite tricky. We will have to adapt Production Debugging techniques, which involves taking a process dump in production environment only when the problem is happening. Challenge is of course how to get the process dump as this problem only happens intermittently. This is where we can use Peformance Counter's Data collector to trigger a creating a task to generate process dump only when the process reaches a pre-defined threshold for our custom performance counter.
Listing below show the code that will cause the TransactionsCount to go high.
private void SimulateHighTransactionsCount()
for (int i = 0; i < 100; i++)
System.Threading.Thread.Sleep(1000 * 2);
private void IncrementTransactionsCount()
using (PerformanceCounter TransactionCount = new PerformanceCounter())
TransactionCount.CategoryName = "MyApplicationStats";
TransactionCount.CounterName = "TransactionsCount";
TransactionCount.ReadOnly = false;
Once the code above get execute, you can see custom performance counter go high as shown in Figure 3 below.
Figure 3: Performance Monitor
Setting up Data-Collector
In order to get a process dump file in the scenario described above, we will first setup a user-defined Data Collector using Performance Monitor. Let's launch the PerfMon, right click on option for User Defined Data Collector Sets and select New -> Data Collector Set option as shown in figure 4 below.
Figure 4: Performance Monitor
At this point, you will get a dialog where you can name your Data Collector. Make sure you select "Create manually" option.
Figure 5: Create new Data Collector dialog
Next dialog will provide you option for adding your performance counter for this Data Collector. Hit Add button here.
Figure 6: Add Performance Counter in Data Collector
Hitting Add button provides you with a list of performance counter. Select and add our custom performance counter from this dialog as shown in figure 7 below.
Figure 7: Choose custom performance counter
Next dialog provides you ability to set the threshold. As shown in figure 8, we want to use 25 as TransactionsCount threshold for generating the dump file.
Figure 8: Defined threshold
On next dialog, just hit "Save and close".
Figure 9: Save your work.
Now there are couple of adjustments that we would like to do in this newly created Data Collector. So click on the properties for this data collector, and you will get a dialog with three tabs, Alerts, Alert Action and Alert Task. First thing we want to do is adjust sample interval in the Alerts tab. For this example, I am adjusting it to 10 seconds, however, for a production environment this will not be an appropriate value. You may not want to monitor your counter this frequently and be better off using a higher value.
Figure 10: Use an appropriate sample interval
Next thing you want to do here is on the Alert Task Tab. You would like to setup the name of the task that will be executed whenever this threshold get hit. The name that you use here has to match exactly with the name that will be used in next step where we will create a task using Windows Task Scheduler.
Figure 11: Setup the task that you like to execute when threshold is reached.
Windows Task Scheduler
In order to configure the task that will be executed each time our performance counter threshold is reached, we will launch Windows Task Scheduler and Create a task.
Figure 12: Create Task.
Next we will setup the Action. In our case, it will be commands that will get executed.
Figure 13: Setup Action
The contents of CreateDump.cmd as follows.
CD C:\"Program Files"\"Debugging Tools for Windows"
ADPlus -hang -pn CustomPerformanceCounter.exe -o C:\Test
Logman -stop TransactionsSpike
We are simply running ADPlus in hang mode. This mean that as soon as ADPlus command is run it will attach to the process name CustomPerformanceCounter (configured using -pn switch) and will generate a process dump file in C:\Test folder. Note that we are stopping the TransactionsSpike data collector right after ADPlus is run, otherwise it will continue to keep running the task which in turn will continue to write the dump file. Given the potentially big size of dump files, this is not something you want to have in your production environment.
Our setup is all complete now. We just have to go back and start the data-collector in the Performance monitor as by default it is created in the the stop mode. Now as soon as you start the data collector and the threshold is hit, you should be able see the dump file. Once you have the dump file, you can use any tool like WinDbg or DebugDiag to analyze dump file and find the root cause of the problem.
In this article, I demonstrated how you can automate process dump file generation based upon custom performance counter threshold using a combination of ADPlus, Performance counters and Windows Task Manager. I hope you find this technique useful.