Click here to Skip to main content
15,867,453 members
Articles / Programming Languages / Visual Basic
Article

A Peek into the Enterprise Instrumentation Framework

Rate me:
Please Sign up or sign in to vote.
4.26/5 (17 votes)
30 Aug 200312 min read 124.7K   51   18
This article takes a look at the Microsoft Enterprise Instrumentation Framework and details how shortcomings of conventional instrumentation techniques are overcome using this framework

Introduction

It is needless to say that Tracing and Instrumentation are a very important facet of any application. The term Instrumentation refers to the ability to monitor or measure the level of a product's performance and to diagnose errors. In terms of programming this may relate to code tracing, debugging techniques, performance counters, event logs and so on (MSDN). The .NET framework includes the Trace and the Debug classes which provide the above mentioned aspects of instrumentation and have been very valuable to application development. Unfortunately, these classes have various shortcomings, which does not make them viable for certain situations or types of applications. To tackle the issues of conventional instrumentation techniques, MS has come up with an comprehensive framework called the Enterprise Instrumentation Framework (let's call it EIF in short). In this article, I shall elaborate what EIF is, and how the shortcomings of the Trace and Debug classes are overcome using the EIF.

Background

Before going forward, I would like to recommend two must read articles regarding Trace and Debug classes and their drawbacks.

What is EIF ?

The Enterprise Instrumentation Framework provides a very simple, yet extensible framework for instrumenting applications. Most importantly, the EIF provides a "Unified API" which helps integrate with already existing frameworks like WMI, Event Log etc. Also, as the name suggests, this framework is suitable for enterprise scale applications. That said, this framework not only enables instrumenting applications on one machine, but even those, which are distributed across several machines.

Let us first understand what EIF comprises of, and then, we can do a comparative analysis with existing instrumentation techniques. The following section describes the basic elements of EIF:

Event Sources

Event sources represent sources from which events occur. Typically, these may be represented by a particular application itself or specific parts or components of an application. The EIF provides three types of Event Sources:

TypeDescription
SoftwareElementThis type is used to control instrumentation behavior at a granular level. What developers can do is, split the application into a number of sources such that event raising can be independently controlled.
ApplicationEvents that are raised without specifying an event source use the Application event source. In this case, Enterprise Instrumentation internally defines a SoftwareElement event source to represent the instrumented application. This event source is given the reserved name of Application.
RequestAccording to me, Request tracing is the best feature provided by the EIF. Using Request tracing, we can instrument an application even if it spans process or machine boundaries. To achieve this, developers would create a request event source and specify the beginning and end points of the request in the application code, which causes Enterprise Instrumentation to automatically manage the context of that request. I would discuss more about this in the later sections.

Event Sinks

Event sinks basically govern the destination store for the event data. Enterprise Instrumentation provides three standard event sinks. You can use Standard event sinks or Custom Event sinks. The standard event sinks provided are WMIEventSink, TraceEventSink and the LogEventSink

Event SinkDescription

TraceEventSink

This event sink writes events to a to a Windows event trace log file if a trace session is active. This tracing mechanism is suitable for the higher-frequency eventing, which can generate hundreds or even thousands of events per second in a running application.

LogEventSink

This event sink outputs the event to the Windows event log and is suitable for lower-frequency events such as errors, warnings, or high-level audits.
WMIEventSink

This event sink outputs the event to Windows Management Instrumentation (WMI). This is the slowest of the standard eventing mechanisms on Windows 2000 systems, and therefore should be used primarily for infrequent or high-visibility events.

Custom Event Sinks

A developer can leverage preexisting log formats and message routing mechanisms, such as Microsoft Message Queue (MSMQ) to create a custom sink. A developer can use either of the following two methods to implement custom event sinks:

  • Derive a new class from the EventSink abstract base class.

  • Derive a new class from the EventSink abstract base class and implement the IEventContainerSink interface.

Event Schema

Event schema defines the set of events which an application can raise. Implementation-wise, the event schema represents a set of classes that the EIF provides. Again, there are two event schemas: Standard and Internal. Standard events are those which a developer explicitly raises and include events like errors, audits, administrative events, and diagnostic trace events. Internal events are fired automatically by the EIF itself (mainly related to Request Tracing).

We can also extend the standard event schema by creating events of our own. The standard event classes provide certain base classes which can be inherited to create custom event classes.

Given below are some simple examples of using Event classes to raise events. The event schema provides an exhaustive set of events and methods which one can use. There is no point detailing all these here, the EIF documentation is more than apt for that.

VB.NET
TraceMessageEvent.Raise("A simple trace event")
AuditMessageEvent.Raise("A simple audit event")

Configuration

We have seen what Event Source, Event Sink and Event Schema means in the previous section. In this section, we shall see how we relate these in an application through configuration. Each application should maintain a configuration file for storing EIF settings and this, by default is called EnterpriseInstrumentation.config. This file can be created programmatically using the Configuration API or by manually editing it. Given below are the typical configuration sections:

Event Categories

In this section, we document what types of event classes are grouped into named categories. This helps to logically group certain event classes in an application. For example, creating a category by name All Events for the type System.Object would include all events (since all classes are implicitly derived from System.Object) in that category. Here's an example:

XML
<eventCategories>
  <eventCategory name="All Events" 
       description="A category that contains all events.">
    <event type="System.Object" />
  </eventCategory>
</eventCategories>
Event Sinks

This section documents all event sinks used by the application and their corresponding parameters. The example shown below declares a TraceEventSink by name traceSink and session name has been configured as TraceSession.

XML
<eventSinks>
  <eventSink name="traceSink" 
     description="Outputs events to the Windows Event Trace." 
     type="Microsoft.EnterpriseInstrumentation.EventSinks.TraceEventSink">
    <parameter name="sessionName" value="TraceSession" />
  </eventSink>
</eventSinks>
Event Sources

This section lists all the Event sources in the application. The example given below documents a request event source by name MyRequest.

XML
<eventSources>
  <eventSource name="MyRequest" type="request" 
    internalExceptionHandler="ignore" 
    description="A request event source." />
</eventSources>
Filter

This section binds a Event Category to one or more Event Sinks. The configuration shown below associates All Events event category to the TraceEventSink and the LogEventSinks respectively. That is, any event raised will automatically be logged to the Windows Event trace log and the Event log.

XML
<filters>
  <filter name="defaultSoftwareElementFilter" 
    description="A default filter for the Software Element event sources.">
  <eventCategoryRef name="All Events">
  <eventSinkRef name="traceSink"/>
  <eventSinkRef name="logSink"/>
  </eventCategoryRef>
  </filter>
</filters>

FilterBindings

This section binds an event source to a filter. In the example given below, the Application event source is bound to the defaultSoftwareElementFilter filter.

XML
<filterBindings>
  <eventSourceRef name="Application">
    <filterRef name="defaultSoftwareElementFilter" />
  </eventSourceRef>
</filterBindings>

Here's the big picture: An application creates Event sources, maps event classes to categories and uses one or more Event sinks. Using filters, an application is configured to route certain categories of events to certain event sinks and Event sources are mapped to these filters using filter bindings. The figure shown below roughly illustrates these relationships.

Relation between elements in EIF

The application uses classes of the event schema to raise certain events and based on the configuration, the event data would find its way into one or more event sinks. Event sinks can finally be queried for data, which can be used to analyze the application performance or behavior.

EIF to the rescue

This section has to be the most interesting one of this article. Here, I shall discuss various shortcomings of conventional instrumentation techniques (Trace and Debug classes) and how EIF aids in overcoming those. I shall also enunciate some of the features provided by EIF, which I found very interesting and useful.

Firstly, the Trace and Debug classes are not inheritable. Therefore, to implement a custom trace functionality, we need to adopt containment as a technique. Here, by containment I mean, wrap the Trace class in your class implementation and add your functionality over it. This implementation can be seen in Marc's article. The EIF event schema, on the other hand is completely extensible. The framework provides a lot of Base classes which can easily be extended. For example, some of the provided base event classes are BaseEvent, CommonEvent, DiagnosticEvent, TraceEvent and so on (The BaseEvent sits at the top of the hierarchy).

The second problem with the trace class is that, you cannot associate different trace messages with different trace listeners. For example, (as mentioned in Vagil's article) you cannot assign different severity thresholds to trace listeners. So, the trace output is same for all listeners. In EIF, you can categorize different event classes into categories and define filters to determine which event sink the messages are routed to. For example, let's consider the InternalErrorMessageEvent and ExternalErrorMessageEvent classes to be categorized into a category called Errors and we can have these events route to the Event log. On the other hand, we can have AdministrativeEvent and AuditEvent classes categorized into a category, say Audit, which can be configured to route events to the Trace log. Therefore, by configuring appropriate filters, we can bifurcate different event data to go to different sinks, something that the trace class and trace listeners definitely lacked.

The third and major drawback with Trace and related classes is the difficulty/inability to do distributed request tracing. By distributed request tracing, I mean the ability to trace an application that may span several processes or several machine boundaries for that matter. If you recollect, the TraceListener collection added (through the Trace.Listeners.Add method) to one process/appdomain is not available directly to another process/appdomain. Therefore, by default, the trace context does not flow from one process to another. To achieve some amount of correlation between trace messages written in different processes, we might end up writing custom trace listeners that write trace output to common stores like SQL Server or MSMQ and use Trace categories to correlate trace messages. Even this method is not completely fool proof in itself. Probably, we may have to use a complex remoting infrastructure for this. Bottom line is, we don't have distributed request tracing "out of the box". Enter EIF, this feature is inherently provided and it is a remoting infrastructure that is used; and the request information is stored in the LogicalCallContext class, which flows along the path of execution, across process and machine boundaries.

Thus far, we saw some features of EIF that were a solution to shortcomings of the Trace class. Now, I shall take you through some features which I think are really cool and helpful. Foremost among them, are certain information fields populated by the event classes. Usually, when we write trace statements in code, we inevitably, end up writing code to capture some information like thread identity, operation execution time, ProcessID etc. The EIF event classes provide a wealth of such information without any coding effort from our side. Each event class provides certain useful fields which are grouped into named property groups. Certain properties are populated by default. For example, a group called PopulateRequestInfo is populated by all events by default. This group includes information like RequestSequenceNumber ( which can be used to track the order of requests), RootRequestInstance (to track the root request) and many more. Certain property groups, on the other hand, are not populated by default. An example for the same is PopulateComPlusInfo, which contains COM+ related information like ActivityID, TransactionID and so on. EIF provides us an extra level of control by providing us an option to decide which property groups are populated and which are not, for a particular event source. In the configuration snippet given below, PopulateWindowsSecurityInfo is enabled for the SecureRequestTrace event source.

XML
<eventSource name="SecureRequestTrace" type="request" 
  internalExceptionHandler="ignore" >
  <eventSourceParameter name="PopulateWindowsSecurityInfo" value="true" /> 
</eventSource>

Another good feature is that of the project installer. Immediately following the imports directives, add the following new empty installer class that inherits from ProjectInstaller. Define it with the RunInstaller attribute set to true.

VB.NET
<RunInstaller(true)> _ 
public class MyProjectInstaller 
    Inherits ProjectInstaller 

After compiling the application, run the installutil.exe on the application executable. Voila! You would see that the EnterpriseInstrumentaion.config is automatically created with all configuration information already documented.

Finally, I was really impressed by the Trace Session Manager. The trace session manager runs as a windows service and is primarily responsible for maintaining trace sessions. Each trace session includes a trace log file and all the event classes which have the event sink configured as the trace log sink end up writing trace data to this log file. (To facilitate this, we just have to make sure that the sessionName parameter of the of the trace log event sink in the EnterpriseInstrumentation.Config matches the name of the configured trace session in the TraceSessions.config). I have already mentioned one advantage of trace logs: they are suitable for high frequency eventing. The other is more of a personal preference. I really prefer analyzing trace data from a trace log file using TraceLogReader API over analyzing the same from an event log. The latter, I thought, requires more plumbing).

Conclusion

The EIF is definitely a great framework for instrumentation, especially for enterprise scale applications and is a definitive improvement over the Trace and Debug classes. The EIF provides easy to use API and also seamlessly integrates with VS.NET. Applications instrumented using EIF are easy to configure and also do not incur much performance degradation as a result of instrumenting the code. Therefore, I would like to conclude, asserting that, there is more than one compelling reason for a development team to adopt EIF as their chief instrumentation strategy in a .NET application.

Unfortunately, in my opinion, the EIF, though being a rich framework is not so well advertised by MS. EIF is only available through MSDN Universal subscriber downloads. The EIF documentation provided with the EIF framework is one of the very few good resources I came across. The other resource is the MSDN TV on EIF and is available here.

Even to this day, I feel EIF still remains to be an unearthed treasure in the MS technology stack!

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


Written By
Web Developer
India India
I am a software developer and have worked on Microsoft technologies for about five years now. I have always been fascinated by Microsoft technologies and with the advent of .NET , this fascination has reached new heights. I take a lot of interest reading technical articles and equally enjoy writing them. I really like to be called a .NET junkie and will surely try to live up to this name Smile | :)

I am .NET MVP and have also completed MCAD, MCSD(VS 6), MCDBA (SQL Server 2000), MCSA (Win 2K) and MCTS (Distributed Apps) certfications.

Comments and Discussions

 
QuestionCustomized Event Sink Pin
geetuadroit6-Nov-06 19:55
geetuadroit6-Nov-06 19:55 
QuestionRookie Question... Pin
Larry Bornacelli22-Nov-05 8:59
Larry Bornacelli22-Nov-05 8:59 
GeneralProblem With Viewing Counters Pin
Football18-Jan-05 10:28
Football18-Jan-05 10:28 
GeneralCustom Event Source Pin
umeshb2-Dec-04 16:07
umeshb2-Dec-04 16:07 
GeneralA sinking feeling Pin
HumorMe22-Apr-04 6:26
HumorMe22-Apr-04 6:26 
GeneralRe: A sinking feeling Pin
CAmelinckx22-Apr-04 9:19
CAmelinckx22-Apr-04 9:19 
GeneralRe: A sinking feeling Pin
HumorMe22-Apr-04 9:34
HumorMe22-Apr-04 9:34 
GeneralRe: A sinking feeling Pin
HumorMe22-Apr-04 9:40
HumorMe22-Apr-04 9:40 
GeneralRe: A sinking feeling Pin
CAmelinckx22-Apr-04 10:18
CAmelinckx22-Apr-04 10:18 
GeneralRequest Tracing limitation Pin
Joao Morais11-Dec-03 0:31
Joao Morais11-Dec-03 0:31 
GeneralAvailable to Everyone Now Pin
mcpell12-Nov-03 5:36
mcpell12-Nov-03 5:36 
GeneralRe: Available to Everyone Now Pin
Manoj G12-Nov-03 16:27
Manoj G12-Nov-03 16:27 
GeneralRe: Available to Everyone Now Pin
Harsh50029-Dec-04 18:50
Harsh50029-Dec-04 18:50 
QuestionRe: Avail to All -- 10/26/2003 vs march-07 ? Pin
pg--az6-Mar-07 17:06
pg--az6-Mar-07 17:06 
GeneralA clash of acronyms (EIF) ! Pin
Manoj G1-Sep-03 20:13
Manoj G1-Sep-03 20:13 
GeneralGreat article! Pin
Marc Clifton31-Aug-03 15:27
mvaMarc Clifton31-Aug-03 15:27 
GeneralRe: Great article! Pin
Manoj G1-Sep-03 3:53
Manoj G1-Sep-03 3:53 
Hi Marc,

The question about the bad availabilty of EIF is really debatable. I suppose (from one of the explanations I got)is that EIF is targeted mostly at high-end enterprises, who is most cases are MSDN Universal Subscribers too. I plan to open this thought in the private MVP news group and lets see what comes of it..

The second question is actually interesting. I agree with you regarding this matter. I am for instrumentation cos I feel there is nothing much to loose by instrumenting. It is always a value add. Generally, the developer need not invest a lot of time coding for instrumentation; the effort I suppose will be miniscule as compared to the actual source itself. And yes, tracing can reveal the true face of the application, which may be obscured during a unit test ..Smile | :)


Thanks again,

Manoj G [.NET MVP]
GeneralRe: Great article! Pin
Mivano21-Oct-03 4:48
Mivano21-Oct-03 4:48 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.