Solving Problems of Monitoring Standard Output and Error Streams of a Running Process

• Download source code - 24.8 KB

Introduction

The System.Diagnostic.Process class allows the execution (or spawning) of other programs from within a .NET application. At times, it may be necessary for a program to monitor the text output of a running process.

This article highlights some of the problems encountered when monitoring output streams via the System.Diagnostic.Process class, and presents a class solution to work around these issues.

At the end of the article, there are several references where you can find more in depth information on some of the topics discussed.

Background: Standard Error, Standard Output, Standard Input Streams

The UNIX Operating System was the first to establish a standard output (stdout), standard error (stderr), and standard input (stdin) stream mechanism. In the spirit of UNIX, these three streams could be treated like files and accessed with standard file read/write functionality. The standard output stream is primarily used for normal data reporting, the standard error is used to report errors or warning information, and the standard input is used to send input text to the program.

Microsoft Windows maintained this philosophy for processes. GUI processes are capable of writing to standard error/output, but this is rarely utilized except for debug type of messages.

Why Monitor Output Streams?

It may be necessary for one program to utilize functionality contained in another software. Often this is done by using an application programming interface (API), Web Services, .NET assemblies, and COM objects. In a perfect world, all software would provide easy to use, fully documented interfacing solutions, but in reality, this is sometimes not the case. It may be that there is not an interface solution for a particular command-line program, but by interacting with the running programs via its output streams, a reasonable level of integration can be achieved.

The Mono project has provided the capability to run on multiple UNIX type platforms, where command line programs are still heavily used. By being able to monitor output and send input to the processes, integration should be fairly straightforward with these command line programs.

Summary of Steps to Monitoring Output Via Standard Methods

(See MSDN documentation for additional explanation)

• Create a System.Diagnostic.ProcessStartInfo object, and set the UseShellExecute property to false, and the RedirectStandardInput, RedirectStandardOutput, RedirectStandardError properties to true.
• Call System.Diagnostic.Process.Start() passing in the pre-initialized ProcessStartInfo object to that method.
• To read output asynchronously (do not block waiting of output), add an event handler to the Process.OutputDataReceived event and call Process.BeginOutputReadLine(). The handler will receive text when the process writes to its standard output stream.
• To read output synchronously (block until process writes text to output stream), call the Process.Read(), Process.ReadLine(), or Process.ReadToEnd() methods.

Two Problems Encountered When Monitoring Output Streams

(See MSDN documentation additional explanation)

1. For synchronous read operations, a deadlock condition may occur when the parent process is waiting on the child process to write text and the child process waits on the parent process to read its text.
2. For asynchronous read operations, Process.OutputDataReceived and Process.ErrorDataReceived are only notified after a newline character is read. This could delay, or prevent, the notification of output text being written (at least until more text is written with a newline terminator). This situation happens with the Windows command line interpreter, "cmd.exe", where the command prompt is written without a newline character and the program is waiting for user input. For example, cmd.exe will write "C:\>" without a newline terminator.

Solutions to Stream Reading Problems

The MSDN documentation suggests having two threads - one for reading stdout and the other for reading stderr. The solution presented implements this suggestion, and additionally solves the problem of blocking on a stream read operation - waiting on a newline character.

The ProcessIoManager class is introduced in this project to address the existing problems of reading output streams. It implements two separate reader threads to monitor both the stdout and stderr streams in the background, and notifies via event handlers when text has been read.

The following pseudo code summarizes using ProcessIoManager:

// Create and initialize ProcessInfo structure 
System.Diagnostics.ProcessInfo myProcessInfo = 
       < create and initialize ProcessInfo structure > ;
// Create and start a new process 
System.Diagnostics.Process myProcess = 
       System.Diagnostics.Process.Start ( myProcessInfo ) ;
// Create a new process manager to monitor and notify of output 
ProcessIoManager processIoMgr = new ProcessIoManager( myProcess ) ; 
// Add event handlers in order to be notified of stdout/stderr text read: 
processIoMgr.StderrTextRead += new StringReadEventHandler(this.OnStderrTextRead);
processIoMgr.StdoutTextRead += new StringReadEventHandler(this.OnStdoutTextRead); 
// Tell the manager to start monitoring the output 
processIoMgr.StartProcessOutputRead() ;
// Add code to the OnStderrTextRead() and OnStdoutTextRead() methods
// to handle the text that has been read from the streams. 
//STOP output monitoring (disposes of the reader threads) 
processIoMgr.StopMonitoringProcessOutput() ;

Pseudo code for threads monitoring and reading stdout/stderr streams

(See the ProcessIoManager.ReadStream(), ReadStandardOutputThreadMethod(), ReadStandardErrorThreadMethod() methods for the implementation of this pseudo-code)

• Clear output buffer
• Perform synchronous read of 1 character on stream (blocks on read until character is read)
• Obtain a synchronization lock, blocking the other stream until all of the current stream is processed
• Add the single character to output buffer:

while ( there are characters in output stream to be read) 
{
    Read single output character Append character to output buffer 
    if ( character was a newline character) 
    { 
        Notify event handlers and pass in output buffer 
        as a string Clear output buffer 
    } 
}

• Notify event handlers of any remaining text in the output buffer
• Clear output buffer

Summary of Project Code

The example project consists of a single Windows Form (MainForm) that contains a text box control, CmdWindowBoxSync, that uses the ProcessIoManager class to implement a command line window that can be used directly on a Windows Form. The control displays the stdout/stderr text, as well as lets the user type in text to be sent to the stdin process stream. CmdWindowBoxSync will notify listeners of stdout/stderr reads via the StdoutTextRead and StderrTextRead events. The command to execute is defaulted to "cmd.exe" and is executed by pressing the "Run And Monitor Process" button. A command line prompt will appear in the interactive text window:

Caveats of the Current ProcessIoManager Stream Reader Threads

The stdout/stderr threads obtain a lock so that they can synchronize reading, so that all text is read from the stream before it relinquishes the lock. This might cause a problem where, for example, some stderr text is interspersed with many lines of stdout text. If this happens, the stderr text may not be received in the correct sequence.

Ideas on Putting Some Pieces Together: Command Line Program Automation

Now that stdout/stderr streams can be simultaneously monitored, command line tasks can now be automated. Instead of a user typing input, a program could be written to monitor and parse program output, formulate an appropriate text response, then send that response to the running process via the stdin stream.

This project was not meant to be a complete dissertation or an all-in-one solution to all problems with output streams, but meant to present a simple project to be used as a building block to solve larger problems.

Happy coding :)

Additional References

• Interacting with Shell Applications (Made Easy)" by "thund3rstruck" (http://www.codeproject.com/KB/dotnet/FunWithProcesses.aspx) - This is an outstanding and well written article about using the System.Diagnostics.Process class.
• System.Diagnostic.Process Microsoft MSDN documentation - http://msdn.microsoft.com/en-us/library/system.diagnostics.process.aspx
• System.Diagnostics.ProcessInfo Microsoft MSDN documentation - http://msdn.microsoft.com/en-us/library/system.diagnostics.processstartinfo.aspx
• MONO Project .NET Multi Platform - http://www.mono-project.com

History

• March 2011 - Curt C. - Initial submission of article and sample project.