Click here to Skip to main content
Click here to Skip to main content

ProcessCommunicator

, 5 Apr 2010 CPOL
Rate this:
Please Sign up or sign in to vote.
A class that allows my CommScript class to "'drive' a command line utility"

Introduction

This is an implementation of an IScriptableCommunicator that can interact with a console application by running it in a Process and redirecting its standard streams.

Background

As I was working on my CommScript[^] and TelnetSocket[^] articles, I realized that a System.Diagnostics.Process with its standard streams redirected could be wrapped in a class that implements IScriptableCommunicator and a program could then execute scripts against a console application (not a Windows application) running locally.

As luck would have it, just a few hours later, a message was posted in the C# forum asking:
How do I 'drive' a command line utility beyond executing a single process / command?[^].
That prompted me to actually proceed with developing this class.

IScriptableCommunicator

To work with CommScript, a class must implement the IScriptableCommunicator interface. I included it in the other two articles, and here I'll include it again:

public delegate void DataReceived ( string Data ) ;

public delegate void ExceptionCaught ( System.Exception Exception ) ;

public interface IScriptableCommunicator : System.IDisposable
{
    void Connect   ( string Host ) ;
    void WriteLine ( string Data , params object[] Parameters ) ;
    void Write     ( string Data , params object[] Parameters ) ;
    void Close     () ;

    System.TimeSpan ResponseTimeout { get ; set ; }

    System.Text.Encoding Encoding { get ; set ; }

    string LineTerminator { get ; set ; }

    event DataReceived    OnDataReceived    ;
    event ExceptionCaught OnExceptionCaught ;
}

ScriptableCommunicator

I have made a few changes to the ScriptableCommunicator abstract class since writing the other two articles. Here is how it now stands:

public abstract class ScriptableCommunicator : IScriptableCommunicator
{
    private System.Text.Encoding encoding       ;
    private string               lineterminator ;

    protected ScriptableCommunicator
    (
        System.TimeSpan      ResponseTimeout
    ,
        System.Text.Encoding Encoding
    ,
        string               LineTerminator
    )
    {
        this.ResponseTimeout = ResponseTimeout ;
        this.Encoding        = Encoding ;
        this.LineTerminator  = LineTerminator ;

        this.Timer           = null ;

        return ;
    }

    public abstract void Connect ( string Host ) ;
    public abstract void Write   ( string Data , params object[] Parameters ) ;
    public abstract void Close   () ;

    public virtual void
    WriteLine
    (
        string          Format
    ,
        params object[] Parameters
    )
    {
        this.Write ( Format + this.LineTerminator , Parameters ) ;

        return ;
    }

    public virtual System.TimeSpan ResponseTimeout { get ; set ; }

    public virtual System.Text.Encoding Encoding
    {
        get
        {
            return ( this.encoding ) ;
        }

        set
        {
            if ( value == null )
            {
                throw ( new System.InvalidOperationException
                    ( "The value of Encoding must not be null" ) ) ;
            }

            this.encoding = value ;

            return ;
        }
    }

    public virtual string
    LineTerminator
    {
        get
        {
            return ( this.lineterminator ) ;
        }

        set
        {
            if ( value == null )
            {
                throw ( new System.InvalidOperationException
                    ( "The value of LineTerminator must not be null" ) ) ;
            }

            this.lineterminator = value ;

            return ;
        }
    }

    protected virtual System.Timers.Timer Timer { get ; set ; }

    public event DataReceived OnDataReceived ;

    protected virtual void
    RaiseDataReceived
    (
        string Data
    )
    {
        if ( this.Timer != null )
        {
            this.Timer.Stop() ;
        }

        if ( this.OnDataReceived != null )
        {
            this.OnDataReceived ( Data ) ;
        }

        if ( this.Timer != null )
        {
            this.Timer.Start() ;
        }

        return ;
    }

    public event ExceptionCaught OnExceptionCaught ;

    protected virtual void
    RaiseExceptionCaught
    (
        System.Exception Exception
    )
    {
        if ( OnExceptionCaught != null )
        {
            OnExceptionCaught ( Exception ) ;
        }

        return ;
    }

    public virtual void
    Dispose
    (
    )
    {
        this.Close() ;

        return ;
    }
}

ProcessCommunicator

This class, of course, derives from ScriptableCommunicator and implements the required methods.

Fields and Constructor

The fields of the class hold references to the Process and two threads for the asynchronous reads of the output and error streams. The constructor sets the default timeout, encoding, and line terminator.

public sealed class ProcessCommunicator : PIEBALD.Types.ScriptableCommunicator
{
    private System.Diagnostics.Process process = null  ;
    private System.Threading.Thread    output  = null  ;
    private System.Threading.Thread    error   = null  ;
    private bool                       abort   = false ;

    public ProcessCommunicator
    (
    )
    : base
    (
        new System.TimeSpan ( 0 , 1 , 0 )
    ,
        System.Text.Encoding.ASCII
    ,
        "\r\n"
    )
    {
        return ;
    }
}

Connect

Connect sets up the Process, Threads, and Timer (if requested) and starts them.

Rive is documented here[^].

public override void
Connect
(
    string Command
)
{
    if ( this.process != null )
    {
        this.Close() ;
    }

    if ( System.String.IsNullOrEmpty ( Command ) )
    {
        throw ( new System.ArgumentException
            ( "No Command provided" , "Command" ) ) ;
    }

    System.Collections.Generic.IList<string> temp = Command.Rive
    (
        2
    ,
        Option.HonorQuotes | Option.HonorEscapes
    ,
        ' '
    ) ;

    this.process = new System.Diagnostics.Process() ;

    this.process.StartInfo.FileName = temp [ 0 ] ;

    if ( temp.Count == 2 )
    {
        this.process.StartInfo.Arguments = temp [ 1 ] ;
    }

    this.process.StartInfo.CreateNoWindow = true ;
    this.process.StartInfo.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden ;

    this.process.StartInfo.UseShellExecute = false ;
    this.process.StartInfo.RedirectStandardInput =
        this.process.StartInfo.RedirectStandardOutput =
        this.process.StartInfo.RedirectStandardError = true ;

    this.output = new System.Threading.Thread ( this.Reader ) ;
    this.output.Priority = System.Threading.ThreadPriority.BelowNormal ;
    this.output.IsBackground = true ;

    this.error = new System.Threading.Thread ( this.Reader ) ;
    this.error.Priority = System.Threading.ThreadPriority.BelowNormal ;
    this.error.IsBackground = true ;

    this.process.Start() ;

    this.output.Start ( this.process.StandardOutput.BaseStream ) ;

    this.error.Start ( this.process.StandardError.BaseStream ) ;

    if ( this.ResponseTimeout.TotalMilliseconds > 0 )
    {
        this.Timer = new System.Timers.Timer
            ( this.ResponseTimeout.TotalMilliseconds ) ;

        this.Timer.Elapsed += delegate
        (
            object                         sender
        ,
            System.Timers.ElapsedEventArgs args
        )
        {
            this.Abort() ;

            throw ( new System.TimeoutException
                ( "The ResponseTimeout has expired" ) ) ;
        } ;

        this.Timer.Start() ;
    }

    return ;
}

Write

(There's no particular reason to use the base streams for reading and writing other than that I can use the Encoding that way.)

public override void
Write
(
    string          Format
,
    params object[] Parameters
)
{
    if ( ( this.process == null ) || this.process.HasExited )
    {
        throw ( new System.InvalidOperationException
            ( "The process appears to be stopped" ) ) ;
    }

    try
    {
        byte[] temp = this.Encoding.GetBytes
        (
            System.String.Format
            (
                Format
            ,
                Parameters
            )
        ) ;

        lock ( this.process.StandardInput )
        {
            this.process.StandardInput.BaseStream.Write ( temp , 0 , temp.Length ) ;
            this.process.StandardInput.Flush() ;
        }
    }
    catch ( System.Exception err )
    {
        this.RaiseExceptionCaught ( err ) ;

        throw ;
    }

    return ;
}

Close

public override void
Close
(
)
{
    this.Abort() ;

    if ( this.process != null )
    {
        this.process.Close() ;

        this.process = null ;
    }

    return ;
}

Abort

private void
Abort
(
)
{
    this.abort = true ;

    if ( this.Timer != null )
    {
        this.Timer.Stop() ;
    }

    if ( this.output != null )
    {
        if ( !this.output.Join ( 15000 ) )
        {
            this.output.Abort() ;

            this.output.Join ( 15000 ) ;
        }

        this.output = null ;
    }

    if ( this.error != null )
    {
        if ( !this.error.Join ( 15000 ) )
        {
            this.error.Abort() ;

            this.error.Join ( 15000 ) ;
        }

        this.error = null ;
    }

    return ;
}

Reader

The Reader method takes the stream to read (output or error) as a parameter.

private const int BufLen = 4096 ;

private void
Reader
(
    object Stream
)
{
    PIEBALD.Types.StreamReader stream = new StreamReader
        ( (System.IO.Stream) Stream , BufLen ) ;

    byte[] buffer = new byte [ BufLen ] ;

    while ( !this.abort )
    {
        int len = stream.Read ( buffer , 0 , BufLen ) ;

        if ( len > 0 )
        {
            this.RaiseDataReceived ( this.Encoding.GetString ( buffer , 0 , len ) ) ;
        }
        else
        {
            System.Threading.Thread.Sleep ( 100 ) ;
        }
    }

    return ;
}

As you can see, the Reader method is performing non-blocking reads on the output and error streams; If there is no data available, then the number of bytes returned is zero (0). If blocking reads were used, the code might not be able to check the state of the abort flag frequently enough and could get blocked indefinitely and have to be aborted -- something that should generally be avoided.

According to the MSDN documentation, System.IO.StreamReader.Read returns: "The number of characters that have been read, or 0 if at the end of the stream and no data was read."
Likewise for System.IO.Stream: "The total number of bytes read into the buffer ... or zero (0) if the end of the stream has been reached."

That means that those methods are supposed to be non-blocking, and indeed, when reading from the output stream, I received zero (0) when there was no data. BUT! While testing, I discovered that that is not the case when reading from the Error stream! When there is no data (and there rarely is) the read would block indefinitely!

The problem is that within the Output stream is a System.IO.__ConsoleStream, but within the Error stream is a System.IO.NullStream. ReadTimeout can't be set on the error stream and none of the available asynchronous techniques were appealing to me (especially the ones that are line-oriented).

StreamReader

When you have a blocking read, but need a non-blocking read, pretty much the only solution is to wrap it in an asynchronous read -- this simply moves the blocking read to another thread so the main thread doesn't get blocked.

My StreamReader class implements asynchronous reads on a Stream. An additional benefit (or drawback, depending on your point of view) is that it uses a List<byte> (rather than an array) as a buffer for the data that has been read from the stream and not yet read by the calling code -- while this is convenient, if the caller doesn't read frequently enough, the buffer could grow quite large (and it doesn't shrink).

Fields and constructor

There are fields to hold the stream to read, the buffer to hold the data that has been read, and a thread that will perform the asynchronous reads. The constructor sets these up and starts the thread.

public sealed class StreamReader : System.IDisposable
{
    private readonly System.IO.Stream                      stream    ;
    private readonly System.Collections.Generic.List<byte> buffer    ;
    private readonly System.Threading.Thread               thread    ;
    private          System.Exception                      exception = null  ;
    private          bool                                  abort     = false ;

    public StreamReader
    (
        System.IO.Stream Stream
    ,
        int              Capacity
    )
    {
        if ( Stream == null )
        {
            throw ( new System.ArgumentNullException
                ( "Stream" , "Stream must not be null" ) ) ;
        }

        if ( !Stream.CanRead )
        {
            throw ( new System.InvalidOperationException
                ( "It appears that that Stream doesn't support reading" ) ) ;
        }

        if ( Capacity < 1 )
        {
            throw ( new System.ArgumentOutOfRangeException
                ( "Capacity" , "Capacity must not be less than one (1)" ) ) ;
        }

        this.stream = Stream ;

        this.buffer = new System.Collections.Generic.List<byte> ( Capacity ) ;

        this.thread = new System.Threading.Thread ( AsyncRead ) ;
        this.thread.IsBackground = true ;
        this.thread.Priority = System.Threading.ThreadPriority.BelowNormal ;
        this.thread.Start() ;

        return ;
    }
}

Read

The Read method is the non-blocking attempt to read data from the buffer:

  • If there is data available in the buffer, then it is copied to the provided array and then removed from the buffer.
  • If not, and an Exception has been encountered by the asynchronous reader, then that Exception is thrown.
  • Otherwise, the method returns zero (0) -- no data available.
public int
Read
(
    byte[] Buffer
,
    int    Offset
,
    int    Count
)
{
    if ( Buffer == null )
    {
        throw ( new System.ArgumentNullException
            ( "Buffer" , "Buffer must not be null" ) ) ;
    }

    if ( Offset < 0 )
    {
        throw ( new System.ArgumentOutOfRangeException
            ( "Offset" , "Offset must not be less than zero (0)" ) ) ;
    }

    if ( Count < 0 )
    {
        throw ( new System.ArgumentOutOfRangeException
            ( "Count" , "Count must not be less than zero (0)" ) ) ;
    }

    int result = 0 ;

    if ( Count > 0 )
    {
        lock ( this.buffer )
        {
            int avail = this.buffer.Count ;

            if ( avail > 0 )
            {
                if ( Count < avail )
                {
                    result = Count ;
                }
                else
                {
                    result = avail ;
                }

                this.buffer.CopyTo ( 0 , Buffer , Offset , result ) ;

                this.buffer.RemoveRange ( 0 , result ) ;
            }
            else if ( this.exception != null )
            {
                throw ( new System.InvalidOperationException
                    ( "The way is shut" , this.exception ) ) ;
            }
        }
    }

    return ( result ) ;
}

AsyncRead

The AsyncRead method simply reads from the Stream and adds any data to the buffer. If an Exception is encountered, it is stored and the method exits.

private void
AsyncRead
(
)
{
    try
    {
        byte[] temp = new byte [ this.buffer.Capacity ] ;

        while ( !this.abort )
        {
            int bytes = this.stream.Read ( temp , 0 , temp.Length ) ;

            if ( bytes > 0 )
            {
                lock ( this.buffer )
                {
                    for ( int i = 0 ; i < bytes ; i++ )
                    {
                        this.buffer.Add ( temp [ i ] ) ;
                    }
                }
            }
            else
            {
                System.Threading.Thread.Sleep ( 100 ) ;
            }
        }
    }
    catch ( System.Exception err )
    {
        this.abort = true ;

        this.exception = err ;
    }

    return ;
}

Using the Code

The ProcessCommunicator class is designed for use with CommScript. The zip file contains all the files necessary to use it as such as well as two little console programs that demonstrate its use.

The included Build.bat file will compile the Master and Slave programs (you may need to set the path to CSC). Master will instantiate a ProcessCommunicator and a CommScript, it will then run a simple script that will execute Slave, interact with it a little, and exit. The output from Slave will then be written to the console.

The meat of Master is:

using
(
    PIEBALD.Types.IScriptableCommunicator pc
=
    new PIEBALD.Types.ProcessCommunicator()
)
{
    PIEBALD.Types.CommScript s = new PIEBALD.Types.CommScript ( pc ) ;

    string script =
    @"
    <Prompt>
    @slave
    >test
    >test
    >test
    $exit
    " ;

    string output ;

    s.ExecuteScript ( script , out output ) ;

    System.Console.WriteLine ( output ) ;
}

Points of Interest

"Did you learn anything interesting/fun/annoying while writing the code?" -- Why, yes, as a matter of fact I did...

  • That the Output and Error streams have different backing stores and that the Error stream violates the contract expressed by the documentation of System.IO.Stream!
  • That some utilities (e.g. FTP and TELNET, at least on WinXP) won't prompt for input when running this way!

StreamReader is a work-around for the first problem. I tried different values for CreateNoWindow and WindowStyle, but none solved the second problem. Nor did I find any command-line parameters for those two utilities that would make them behave properly.

UtilityTester

Also included in the zip file, and built by the Build.bat, is a simple console utility that may help test the console program that you intend to automate. When investigating a utility that you would like to automate, try it with the tester first.

It will take its command-line parameters and cobble up a script to execute.
Syntax: UtilityTester start_command exit_command [ prompt [ command... ] ].

  • Example 1: UtilityTester ftp quit
    • You will see that the prompt does not get displayed.
  • Example 2: UtilityTester nslookup exit
    • You will see that the prompt (>) does get displayed, so try...
  • Example 3: UtilityTester nslookup exit ">"
    • You should see that the $exit isn't sent until after the prompt (>) is detected in the output.
      Note: A single character, like ">", does not make a good prompt.
  • Example 4: UtilityTester "Slave \"What is thy bidding, master? \"" exit "What is thy bidding, master?" "Hello" "World!"
    • This demonstrates providing command-line parameters to the utility being tested and issuing some commands as well.

History

  • 2010-04-03 First submitted

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

PIEBALDconsult
Software Developer (Senior)
United States United States
BSCS 1992 Wentworth Institute of Technology
 
Originally from the Boston (MA) area. Lived in SoCal for a while. Now in the Phoenix (AZ) area.
 
OpenVMS enthusiast, ISO 8601 evangelist, photographer, opinionated SOB, acknowledged contrarian
 
---------------
 
"If you need help knowing what to think, let me know and I'll tell you." -- Jeffrey Snover [MSFT]
 
"Typing is no substitute for thinking." -- R.W. Hamming
 
"I find it appalling that you can become a programmer with less training than it takes to become a plumber." -- Bjarne Stroustrup
 
ZagNut’s Law: Arrogance is inversely proportional to ability.
 
"Well blow me sideways with a plastic marionette. I've just learned something new - and if I could award you a 100 for that post I would. Way to go you keyboard lovegod you." -- Pete O'Hanlon
 
"linq'ish" sounds like "inept" in German -- Andreas Gieriet
 
"Things would be different if I ran the zoo." -- Dr. Seuss
 
"Wrong is evil, and it must be defeated." – Jeff Ello
 
"A good designer must rely on experience, on precise, logical thinking, and on pedantic exactness." -- Nigel Shaw
 
“It’s always easier to do it the hard way.” -- Blackhart

“If Unix wasn’t so bad that you can’t give it away, Bill Gates would never have succeeded in selling Windows.” -- Blackhart

"Omit needless local variables." -- Strunk... had he taught programming
 

 
"We learn more from our mistakes than we do from getting it right the first time."
 
My first rule of debugging: "If you get a different error message, you're making progress."
 
My golden rule of database management: "Do not unto others' databases as you would not have done unto yours."
 
My general rule of software development: "Design should be top-down, but implementation should be bottom-up."

Comments and Discussions

 
QuestionApplication in different directory ? PinmemberXmen W.K.23-Jun-11 16:57 
AnswerRe: Application in different directory ? PinmemberPIEBALDconsult23-Jun-11 17:02 
GeneralRe: Application in different directory ? PinmemberXmen W.K.23-Jun-11 17:03 
GeneralRe: Application in different directory ? PinmemberPIEBALDconsult23-Jun-11 17:10 
GeneralRe: Application in different directory ? PinmemberXmen W.K.23-Jun-11 17:14 
GeneralRe: Application in different directory ? PinmemberPIEBALDconsult23-Jun-11 19:36 
GeneralRe: Application in different directory ? PinmemberXmen W.K.24-Jun-11 5:53 
GeneralExcellent! PinmvpDaniel Vaughan6-Apr-10 0:56 
GeneralRe: Excellent! PinmvpPIEBALDconsult6-Apr-10 5:16 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.141223.1 | Last Updated 5 Apr 2010
Article Copyright 2010 by PIEBALDconsult
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid