![]() |
General Reading »
Book Chapters »
General
Intermediate
Upgrading to Lotus Notes and Domino 7 - Chapter 3 : Domino Domain MonitoringBy Mohan RaphelDomino Domain Monitoring (DDM) allows you to monitor the status of multiple servers in one or more domains, all from a single location. |
VB, Windows, .NET, Visual Studio, Dev
|
|
Advanced Search Add to IE Search |
|
|
|
||||||||||||||||
|
|
|
This chapter, along with the ones that follow, discusses the many new features found in the Domino 7 server. Here is a list of the features:
One of the most significant new Domino 7 features�one that's gotten a lot of attention throughout the early beta programs�is Domino Domain Monitoring (DDM). This feature allows you to monitor the status of multiple servers in one or more domains, all from a single location.
DDM uses a set of preconfigured probes to gather status and process information about the servers being monitored. These probes collect data relating to applications, databases, directories, messaging, the operating system, replication, security, the server, and the Web. Special filters allow you to select the type and level of data recorded by the probes. After this data has been collected, it is consolidated, organized, and processed into easy-to-read summary reports. The data is then entered automatically into the Event Resolution Center (ERC). Each event that is processed and placed into the ERC database has a document link back to the specific monitor that generated the event. The ERC is updated with a status document each time a probe detects an error, or a particular threshold is exceeded. By viewing DDM events recorded in the ERC, you can identify (and in some cases even predict) systemic Domino events. The ERC is automatically created when you start the first server. The ERC database is based on the new template ddm.ntf; by default its file name is ddm.nsf.
By default, results generated by DDM probes are placed in the ERC on each server that runs the probes. You can create a DDM server collection hierarchy to aggregate data from several servers to a single server. By using this collection hierarchy, you can designate that a single server can collect all DDM-based event data (and thus use this single server to monitor multiple servers in your domain). Alternately, your can set up several servers to collect data across a domain.
The following figure shows how you can set up servers to collect DDM data in a worldwide domain. In this case, we use a multi-tiered collection model. The top server in our example is Server A in Los Angeles. This server collects data from itself and three other collection servers located in the USA. Collection Server B, located in a data center in Asia, collects data for itself and two other servers in Hong Kong and Tokyo. Collection Server C collects data from itself and two servers located in London and Paris.

Reported data is generated in each ERC based on its location in the collection architecture. To review data about the London server, an administrator can open the ERC on Server C, where data for the London and Paris servers is stored. It's also possible to view London data by opening the ERC on Server A, which contains all DDM data for all servers in the hierarchy.
This collection hierarchy is possible because each Domino 7 server writes its own probe results into a local ERC replica on each collection server. As a result, the ERC maintains data about its own probes as well as the probe data from every server that is monitored by this server. As you can see from the preceding figure, data is rolled up and pushed into the collection server that represents the next higher level in the tree. This process is managed by Lotus Notes replication. Selective replication formulas are automatically created when you create the DDM server hierarchy. Using this simple technique, you see the rolled up data where you want to see it�for instance, in the figure, data from Hong Kong exists on Server B (the Asia server) and Server A (the top-level server), but not on Server C (the Europe server).
Probes are the internal engines that make DDM work. There are nine types of probes available in Domino 7:
You can select which probes you want to run, and what data these probes collect, through Probe documents. These documents reside in the Events database (events4.nsf). Through Probe documents, you can specify when the probe runs. Many probes can be configured to run on a schedule, on an event, or real-time. The function of the probe will dictate what type of schedule can be executed. For example, if you select the Schedule option, you can choose to run the probe:
For example, if you want the probe to run on the fifteenth day of each month, enter 15.
In some cases (for example, the Security probe), you can enable the probe to run when a particular event occurs, such as when a Person, Server, and/or a Configuration document has changed. This can provide a very quick alert back to an administrator. You can also determine how missed probes are handled�you can ignore the missed probe, run the missed probe on startup, or run it at the next time range.
One very convenient feature is the ability to assign probe events based on server type. For many probes, you can select an option called Special Target Servers, which offers a set of server types, including:
For instance, if you select the type as mail server, the probe will run on all mail servers in your domain.
You can create DDM filters to control the event type and event severity of events generated inside and outside of DDM. These filters determine what data is included in the DDM log file. You can specifically include all DDM events or include/exclude specific type of events. The DDM filter is created in the events4.nsf database. After the filter document has been created, you can determine the following for each filter document:
The Event Resolution Center (ERC) ("See ERC, event resolution center" database (ddm.nsf)) contains the data generated by active DDM probes. When a probe runs, it records all the relevant data that it finds (if any) to a report that is placed in the ERC. This report contains the results of the specific probe, the probable cause that generated the result, suggested solutions for each event, and a link to the probe that was used to generate this event.
The ERC includes seven navigator views and a link to events4.nsf. Each view can help you find and/or diagnose a particular problem. The views are as follows:
As we mentioned earlier, there are nine types of probes. You select the type of probe you want to create in the Probe document:
This probe monitors an agent's schedule and resource usage. It also checks for agent-related conditions and events such as agents that are disabled by the design server task, agent security errors, and agent full-text index errors resulting from search operations.

The Database probe monitors database status and activities such as database compacting errors and design errors, as well as status on the ability to actually open this database. This probe has four different probe types that you can enable:

Note the option to remove error codes from the list of errors that are to be recorded. By default, a number of error codes are automatically ignored, such as "Document has been deleted", "Entry not found in index", "File does not exist", and so on.
The Directory probe is one of our favorites. This powerful probe monitors many different directory functions:

The Messaging probe monitors the Domino-based messaging infrastructure. Features include the ability to monitor SMTP activity, Notes (NRPC) mail routing, and various mail-routing statistics. There are currently ten options that you can choose from:
Mail.Total.Pending, Mail.Dead, Mail.Held, and Mail.Waiting. This lets you monitor the quantity of mail moving through the Domino server. The slack percentage is a representative indication of how the router is processing mail.
POP3 is defined in RFC-1725, and IMAP4 is defined in RFC-1730.
One of the many challenges that an administrator must deal with is the status of system resources. Operating System probes provide a mechanism to alert you about potential problems at the OS level. There are four types of Operating System probes that you can enable: CPU, Disk, Memory, and Network.
| OS |
Statistics monitored |
| AIX |
Processor utilization percentage Processor queue length |
| zOS |
Processor utilization percentage |
| Linux, zLinux |
Processor utilization percentage |
| Solaris |
Processor utilization percentage Processor Queue Length |
| OS400 |
Processor utilization percentage |
| Windows |
Processor utilization percentage |
Each selection can be configured with a high/low threshold based on each statistic percentage.
| OS |
Statistics monitored |
| AIX |
Disk utilization percentage |
| Linux, zLinux |
Disk utilization percentage Disk Service Time (ms) |
| OS400 |
Disk utilization percentage |
| Solaris |
Disk utilization percentage Disk service time (ms) |
| Windows | Disk queue length |
| OS | Statistics monitored |
| AIX |
Scan rate |
| OS400 |
Fault rate formula |
| ZOS |
Available frames Out ready queue length Paging rate |
| Solaris |
Scan rate |
| Windows |
Available physical memory (MB) |
| OS | Statistics monitored |
| AIX |
Network bandwidth utilization percentage Network collision rate percentage |
| Linux; zLinux |
Network collision rate percentage |
| OS400 |
Network bandwidth utilization percentage |
| Solaris |
Network bandwidth utilization percentage Network collision rate percentage |
| Windows |
Network bandwidth utilization percentage |
The Replication probe lets you monitor various replication activities within your Domino domain. Replication probes can be configured to monitor all database replication, or replication on specific databases. There are two options available with this probe: Errors and Replication Check.
You can also monitor push- and/or pull-type replication events.

You can create a Security probe to assess the overall security of servers and databases in your domain. A Security probe can identify a security-related server configuration problem and/or security issues with specific databases.
One significant variable with security probes is how the event severity is assigned. Severity is assigned during runtime and is calculated based on the number of various potential issues found. This severity is a percentage-level score that is generated for each Server Configuration document analyzed. (Consult the product documentation for details about how these percentages are calculated.) The basic percentages are shown in the following table:
| Probe percentage | Severity level |
| 0.00 | Normal |
| < = 50% | Warning (low) |
| > 50% | Warning (high) |
There are five Security probe types: Best Practices, Configuration, Database ACL, Database Review, and Review.

The Server probe monitors the administration process for errors, and reports the errors back to the ERC database. The following administration requests can be monitored:
The Web probe monitors Web-related statistics and events. You can select from two probe types, Web Best Practices and Web Configuration:
After you open the configuration setting for a Web probe, you will notice that you cannot assign severity levels. The severity of an event generated by Web probes is determined using a percentage formula. This score is based on the number of potential problems that are found. After this calculation is complete, a 'severity percentage score' is calculated and logged to the ERC.
Domino has the ability to monitor and execute specific actions based on a large variety of events. These events can be the results of normal processing activities or can be error-type events. Event examples include:
With Notes/Domino 7, the Domino administrator now has several new options that can be triggered by an event. When an event takes place, an administrator now has the option to run an agent, run a program (with new parameters), send a console command to the server, or send a Java controller command.
Domino 7 includes processes called 'event generators'. These generators gather information by monitoring Domino tasks and statistics. Also, there is a built-in probing system that can be enabled and linked into the event generators. Specific conditions and/or thresholds can initialize event generators. Once an event generator has been started, it can pass data into the event monitor task. The event monitor task can be loaded manually at the Domino server console, or can be set to load each time the server is started via the NOTES.INI file, in the servertasks line. Once the event has been passed into the event monitor task, it will be processed against event handler configuration documents in the events4.nsf database. If there are no configuration documents defined, then no actions are executed when an event is passed into the event handler.
The Domino Administrator includes a set of default event generators; these are shown in the Event Generators view of the Monitoring Configuration database (events4.nsf). The following table lists the types of event generators that can be created:
| Event generator | Description |
| Database event generator | Monitors database activity and free space. Monitors frequency and success of database replication. Reports on ACL changes, including those made by replication or an API program. |
| Domino server response event generator | Checks connectivity and port status of designated servers in a network. |
| Mail routing event generator | Sends a mail-trace message to a particular user's mail server and gathers statistics indicating the amount of time, in seconds, it takes to deliver the message. |
| Statistic event generator | Monitors a specific Domino or platform statistic. |
| Task status event generator | Monitors the status of Domino server and add-in tasks. |
| TCP server event generator | Verifies the availability of Internet ports (TCP-based services) on servers and generates a statistic indicating the amount of time, in milliseconds, it takes to verify that the server is responding on the specified port. |
Each event document can have a severity assigned. The severity levels can be assigned to each event as needed. Examples of these are shown below:
| Severity level | Meaning |
| Fatal | Imminent system crash |
| Failure | Severe failure that does not cause a system crash |
| Warning (high) | Loss of function, requiring intervention |
| Warning (low) | Performance degradation |
| Normal | Status messages |
To create a new event, just open the events4.nsf and select New Event Handler. The following screenshot shows how this looks:

There are three tabs: Basics, Event, and Action. The Basics tab provides two basic sections: Server(s) to monitor and Notification trigger. The first section is where the servers to monitor are set. The second section has three choices:
Each choice will display additional choices on the Event tab. The following illustration shows these choices:



The administrator can create an event handler document to specify how to log the event to a specified destination. Also, administrators can prevent events from being logged or handled in any circumstance.
At this point, you might be asking, "What is this scripting stuff that I keep hearing about?" Let's start with an example and a goal. The goal is to send a notification and track ACL notifications in a log database. Use the following steps:
Overall, almost any database format will work. Once the database has been created, you will need to create a form, agent, and a view.
The form can have the following fields defined:
| Field name | Field Data Type | Field Description |
EventText |
TEXT |
Text of event |
TargetServer |
TEXT |
Target server for this event |
EventTime |
TIMEDATE |
Time and date stamp of event |
EventType |
NUMBER |
Type of event |
EventSeverity |
NUMBER |
Severity of event |
EventPrams |
TEXT |
Text parameters in event |
ErrorCode |
TEXT |
Event type error code |
OriginatingServer |
TEXT |
Server that originated the event |
EventSeverityText |
TEXT |
Textual representation to Severity |
EventTypeText |
TEXT |
Textual representation to Type |
Here's a screenshot example of a form:

Create a default view as well. Any view will do; you can customize it as you like.
Next up is the agent. Below is a sample agent that you can base your agents on. Name the agent EventAgent. This agent uses the DocumentContext method. This method is a Read-only property. Basically, an in-memory document is created when an agent starts. This method is defined in the NotesSession class, and uses NotesDocument as its data type.
The basic syntax for the DocumentContext is:
To get:
Set notesDocument = notesSession.DocumentContextSub Initialize
Dim session As New NotesSession
Dim doc As NotesDocument
Set doc = session.DocumentContext
Print "Event Text = " & doc.Eventtext(0)
Print "Event Error Code = " & doc.errorcode(0)
' Document Information
Call Doc.Save(True,True)
Set db = session.CurrentDatabase
Set tardoc = db.CreateDocument
tardoc.form = "EventForm"
tardoc.Subject = "Event Information"
tardoc.EventText = doc.Eventtext(0)
tardoc.EventPrams = doc.EventPrams(0)
tardoc.ErrorCode = doc.Errorcode(0)
tardoc.EventSeverity = doc.EventSeverity(0)
tardoc.EventSeverityText = doc.EventSeverityText(0)
tardoc.EventTime = doc.EventTime(0)
tardoc.EventType= doc.EventType(0)
tardoc.EventText = doc.Eventtext(0)
tardoc.EventTypeText = doc.EventTypeText(0)
tardoc.OriginatingServer = doc.OriginatingServer(0)
tardoc.EventErrorcode = doc.errorcode(0)
Call tardoc.Save( True, True )
Print "Agent Complete ! ! ! "
End Sub
Don't forget to:
Open the events4.nsf database on the server and select New Database Event Generator. Enable the event generated to monitor ACL changes for names.nsf. Here's an example screenshot:

Notice the TSPD-6FR4CG string in the screenshot. You will see a similar string, and it can be used as a reference point for the event handler.
In events4.nsf, you can also create the event handler document. Using the flowchart as a reference, select New Event Handler. We'll now take a look at the changes/settings to be made.
Select and complete the following fields (refer to the first screenshot under the Event Notification Using an Agent section):
Select the event using the reference string from the event generator. Once this is selected, you will be able to see details on that document.
Select Run an Agent. Complete the following fields:
EventAgent (from the preceding example)
On the Action tab, enable the event handler and set a schedule if needed.
It is now time to test your event handler, agent, and event generator. Open the names.nsf database on the server where you have enabled each of the event handlers and the generator. Make a simple change to the ACL of the names.nsf database. As soon as this happens, you should see a display on the server console and a document created in your database. An example is shown in the following screenshot:

In this chapter, we have taken a detailed look at Domino Domain Monitoring (DDM), one of the major new administration features introduced in Domino 7. We reviewed the set of pre-configured probes that gather status and process information (applications, databases, directories, messaging, the operating system, and so on) about the servers being monitored. We discussed the Event Resolution Center (ERC), where processed events are placed. We also talked about event monitoring, Domino 7's ability to monitor and execute specific actions based on a large variety of events. These events can be the results of normal processing activities and/or error-type events. The next chapter discusses the Administration process.
| You must Sign In to use this message board. | |||||||||||||||
|
|||||||||||||||
|
|||||||||||||||
|
|||||||||||||||
|
|||||||||||||||
General
News
Question
Answer
Joke
Rant
Admin
|
PermaLink |
Privacy |
Terms of Use
Last Updated: 26 Jan 2006 Editor: Smitha Vijayan |
Copyright 2006 by Mohan Raphel Everything else Copyright © CodeProject, 1999-2009 Web18 | Advertise on the Code Project |