Click here to Skip to main content
Email Password   helpLost your password?
Screenshot - AsyncPowerShell_scr.png

Introduction

My previous article showed how to run PowerShell scripts from C#. That implementation was limited in the sense that it would run scripts synchronously, blocking until the script finished what it was doing. That is fine for short-running scripts, but if you have long-running or even never-ending scripts, you will need asynchronous execution. This article shows just how to do that.

Basic Steps

Here are the basic steps to run a PowerShell script asynchronously:

There are two ways in which you can be notified of the availability of new output data:

The Trouble with Output.WaitHandle

At first hand, Output.WaitHandle seems like a nice option to choose for retrieving the script output data; it provides complete separation between the producer (the PowerShell thread) and the consumer (the output reading thread), unlike the DataReady event which is called directly from the PowerShell thread. But there's a problem: if the consumer isn't fast enough in dealing with the output, the producer will happily continue producing at full speed until it consumes all your memory, or when the script ends. While it seems as though there are provisions in the output queue to limit the maximum amount of memory used (there is a MaxCapacity readonly property), I haven't found a way to actually set this limit.

You may wonder why the consumer would be too slow to process the output. Well, if you're using PowerShell scripting to automate some aspects of your C# program, you'll probably want to display the script output in some way on your user interface. The output of a PowerShell script will easily outrun the refreshing capabilities of any GUI.

Ready for DataReady

Since the DataReady event is called directly from the PowerShell thread, it allows us to throttle back the PowerShell processing speed to the point where it exactly matches the throughput of the consumer. Of course, this means it will reduce the speed of the PowerShell script, but I feel that is the lesser of two evils, in this case.

PipelineExecutor

All of the above steps have been wrapped in a single helper class called PipelineExecutor. This class helps you to easily run a PowerShell script in the background, and also provides you with events to receive the output data of the script. It also 'Invokes' the data to the correct thread, so when the events arrive, you no longer have to perform the dreaded 'InvokeRequired' routine just to display the data. Here's the public interface of the class:

/// Class that assists in asynchronously executing
/// and retrieving the results of a powershell script pipeline.

public class PipelineExecutor
{
    /// Gets the powershell Pipeline associated with this PipelineExecutor
    public Pipeline Pipeline
    {
        get;
    }

    public delegate void DataReadyDelegate(PipelineExecutor sender,
                                           ICollection<psobject> data);
    public delegate void DataEndDelegate(PipelineExecutor sender);
    public delegate void ErrorReadyDelegate(PipelineExecutor sender,
                                           ICollection<object> data);

    /// Occurs when there is new data available from the powershell script.
    public event DataReadyDelegate OnDataReady;

    /// Occurs when powershell script completed its execution.
    public event DataEndDelegate OnDataEnd;

    /// Occurs when there is error data available.
    public event ErrorReadyDelegate OnErrorRead;

    /// Constructor, creates a new PipelineExecutor for the given powershell script.
    public PipelineExecutor
        (Runspace runSpace, ISynchronizeInvoke invoker, string command);

    /// Start executing the script in the background.
    public void Start();

    /// Stop executing the script.
    public void Stop();
}

Using the Code

The following code shows how to create and asynchronously run a PowerShell script using PipelineExecutor:

    ...
    using System.Collections.ObjectModel;
    using System.Management.Automation;
    using System.Management.Automation.Runspaces;
    using Codeproject.PowerShell
    ...

    // create Powershell runspace
    Runspace runSpace = RunspaceFactory.CreateRunspace();

    // open it
    runSpace.Open();

    // create a new PipelineExecutor instance

    // 'this' is the form that will show the output of the script.
    // it is needed to marshal the script output data from the
    // powershell thread to the UI thread
    PipelineExecutor pipelineExecutor =
      new PipelineExecutor(runSpace, this, textBoxScript.Text);

    // listen for new data
    pipelineExecutor.OnDataReady +=
      new PipelineExecutor.DataReadyDelegate(pipelineExecutor_OnDataReady);

    // listen for end of data
    pipelineExecutor.OnDataEnd +=
      new PipelineExecutor.DataEndDelegate(pipelineExecutor_OnDataEnd);

    // listen for errors
    pipelineExecutor.OnErrorReady +=
      new PipelineExecutor.ErrorReadyDelegate(pipelineExecutor_OnErrorReady);

    // launch the script
    pipelineExecutor.Start();

Terminating the script and cleaning up:

pipelineExecutor.OnDataReady -=
  new PipelineExecutor.DataReadyDelegate(pipelineExecutor_OnDataReady);
pipelineExecutor.OnDataEnd -=
  new PipelineExecutor.DataEndDelegate(pipelineExecutor_OnDataEnd);
pipelineExecutor.OnErrorReady -=
  new PipelineExecutor.ErrorReadyDelegate(pipelineExecutor_OnErrorReady);
pipelineExecutor.Stop();
// close the powershell runspace
runSpace.Close();

Please note that to compile the example project, you'll first have to install PowerShell and the Windows Server 2008 SDK. For details, see my previous article.

Error Handling

There are two kinds of errors that you can encounter during execution of a powershell script:

Errors of the first kind will be pushed into the error pipeline. You can listen for those by adding an event handler to PipelineExecutor.OnErrorReady.

To detect and display errors of the second kind (fatal syntax errors) you need to inspect the property Pipeline.PipelineStateInfo.State inside your OnDataEnd event handler. If this value is set to PipelineState.Failed then the property Pipeline.PipelineStateInfo.Reason will contain an exception object with detailed information on the cause of the error. The following code snippet shows how this is done in the example project:

    // OnDataEnd event handler
    private void pipelineExecutor_OnDataEnd(PipelineExecutor sender)
    {
        if (sender.Pipeline.PipelineStateInfo.State == PipelineState.Failed)
        {
            AppendLine(string.Format("Error in script: {0}", sender.Pipeline.PipelineStateInfo.Reason));
        }
        else
        {
            AppendLine("Ready.");
        }
    }

If you want to see the error handling in action, please execute the "Error handling demonstration" script of the example project.

Points of Interest

For people interested in the inner workings of PipelineExecutor, I'd like to point out the private StoppableInvoke method. It waits for the target thread to process the data, thus causing the throttling effect on the PowerShell script. It also avoids potential deadlock problems that would happen if I had simply used ISynchronizeInvoke.Invoke because it is interruptible by way of a ManualResetEvent. This pattern could be useful in other workerthread-to-UI notification situations.

I've also worked on improving the performance of the script execution by separating the 'BeginInvoke' and 'EndInvoke' stages of StoppableInvoke into subsequent DataReady cycles, and this works like a charm... but the resulting output performance is just too close to the consumer performance, causing lag in the user interface. My theory is that .NET likes to give priority to Invoke messages before handling the UI update messages. So, if you get close to 100% of the message loop performance, it will starve the UI updates. Solving this problem probably warrants a different article.

History

You must Sign In to use this message board.
 
 
Per page   
 FirstPrevNext
GeneralWebBase Version
kfandst
18:37 11 Jan '10  
Do you have a VisualWeb version of excuting Powershell Scripts?
GeneralQuestion on the differences in these outputs from your 2 example powershell programs
Member 3272187
11:15 23 Dec '09  
Hello,

I'm using the simple Get-Date cmdlet.

Here is the 2 Different outputs that I get.

HowToRunPowerShell

Wednesday, December 23, 2009 2:00:20 PM

Asynchronous PowerShell Scripting

12/23/2009 1:59:48 PM

Why is this happening.

Thanks,

Mike
GeneralRe: Question on the differences in these outputs from your 2 example powershell programs
jpmik
11:33 23 Dec '09  
Hi Mike,

If you look at the source of the HowToRunPowerShell article you'll see the following line:

pipeline.Commands.Add("Out-String");

So that causes the 'Out-String' command to be appended to your scripts. This lets powershell format the results to a string, according to the powershell defaults. If you remove that line the results will be identical. In comparison, the asynchronous project simply uses an object.ToString() to format the results.

If you use the following script in the asynchronous host, you'll see the first result again:

Get-Date | Out-String
GeneralWindows Service with Class PowerShellExecute
psaid
8:27 15 Jul '09  
Hello,


Firstly I thank you for the excellent work.
Now, I wanted to ask whether it is possible to implement a Windows Service, with the class PowerShellExecute. I tried but could not. I always have an error in:

pipelineExecutor = new PipelineExecutor(runSpace, this, "get-Help");

it can not convert servicebase to ISynchronizeInvoke.

Maybe you can help me.

Thanks in advance,

Paulo
GeneralRe: Windows Service with Class PowerShellExecute
Jean-Paul Mikkers
13:05 15 Jul '09  
Hi Paulo,

In the examples I pass 'this' to the pipelineexecutor constructor because 'this' is a Window which implements ISynchronizeInvoke. In a service you normally don't have a userinterface, so you will probably have to create your own implementation of ISynchronizeInvoke and pass that to PipelineExecutor.

Alternatively (and probably EASIER) is to modify PipelineExecutor.cs so it calls the OnDataReady, OnDataEnd, OnErrorReady events directly, without using the ISynchronizeInvoke. Here's how you do that.. replace the StoppableInvoke() method in the sources by this:

        
private object StoppableInvoke(Delegate method, object[] args)
{
return method.DynamicInvoke(args);
}


Now the ISynchronizeInvoke isn't used anymore so you can just pass null in the constructor.
GeneralRe: Windows Service with Class PowerShellExecute
psaid
13:56 15 Jul '09  
Thanks for your reply.
I try your solution tomorrow morning. I will try and give my feedback here.

Thank you,

Paulo
GeneralRe: Windows Service with Class PowerShellExecute
psaid
3:46 16 Jul '09  
Hello,

Works.

But now i have a prblem.Frown

I have a pipe that creates a session in Outlook Live with a PowerShell.
I have only one session, I can't create multiple sessions.
Now, the problem is that I have multiple clients to execute commands in this session, through sockets.
My problem, is return the right result of the PowerShell Command invoked by a client (socket), to the right invoker client (sokect).

Is this possible using your class (PowershellExecute)...

If not, can you help me!!!

Thanks in advance,


Paulo
GeneralRe: Windows Service with Class PowerShellExecute
caiovalentim
6:57 18 Nov '09  
Hi Jean-Paul,

I've been working on a web version of your script.
That is, a page where one can add some powershell
scripts and it be executed and the output displayed.

Initially I had the same problem that Paulo and I fixed it
with your tip. But I am still having problem to get everthing working. Right now, it executes perfectly when I am debugging but it doesn't show anything when there are no breakpoints.

I thought it could be a problem with this "fix" we're using here. So, I changed your original code to do not call BeginInvoke and EndInvoke but DynamicInvoke(as you said) and it starts to throw an exception about thread invalid access[1].

So, can the same error going in the modified version of your script be affecting my implementation in some hidden way and causing this strange "only work in debug" behavior?

Although I am too noob in C# it looks to me that another option would be implement ISsynchronizeInvoke interface inside my page class. But it seems hard for my current C# skills. Do you have some kind of general advice to do this job in a softer way?

Thank you very much!
Very nice article!

Caio

[1]- "Cross-thread operation not valid: Control 'listBoxOutput' accessed from a thread other than the thread it was created on."
GeneralRe: Windows Service with Class PowerShellExecute
Jean-Paul Mikkers
9:42 18 Nov '09  
In your case I would try this:

- replace all the StoppableInvoke calls by direct calls to the event handlers OnDataReady, OnDataEnd, OnErrorReady
- so, now your application will get all the data, but it's on the wrong thread
- in the application event handlers, put all the data in a Queue, and make sure you lock() the queue when you do
- now on the userinterface thread you need a timer or something that will retrieve the data from that queue (now it's on the correct thread)
- display the data
- done!
GeneralHere's how to get rid of all those Out-String's
Mike Sargent
7:29 13 Sep '08  
This is a great article. It works very well, but I found that formatting is different from PowerShell.
PowerShell normally formats output by default. Here's the simple code to add so your sample will work the same as PowerShell. It's added after the pipeline is created, like the following:

// Create a pipeline and feed it the script text.
pipeline = runSpace.CreatePipeline( command );

// Ensure the output is formatted and sent out the same way the PowerShell console does.
// Taken from http://mshforfun.blogspot.com/2006/07/why-there-is-out-default-cmdlet.html
// Also from http://msdn.microsoft.com/en-us/library/cc136032(VS.85).aspx
pipeline.Commands[ 0 ].MergeMyResults( PipelineResultTypes.Error, PipelineResultTypes.Output );
pipeline.Commands.Add( "Out-Default" );
By adding the Out-Default, output is automatically formatted just like PowerShell.

Thanks,
Mike
GeneralRe: Here's how to get rid of all those Out-String's
Mike Sargent
9:08 13 Sep '08  
Of course, Out-Default writes directly to the host (like it used Write-Host) so you need to specify a host when creating the runspace. If not, the default runspace will just throw exceptions and continue for each item output. Nothing will be output.

Mike
GeneralIDisposable over FormPowerShellSample_FormClosing
dance2die
9:54 27 Aug '08  
First of all, thank you for the great posts(including How to run PowerShell scripts from C#), Jean-Paul

I have a suggestion for the code update:
Wouldn't it be a better idea to implement IDisposable instead of cleaning up (Stopping script and closing runspace) within FormClosing event handler?

Will there be any implications or caveats that need to be dealt with if asynchronous PowerShell resources are freed within a Dispose method?

Thanks.

-- Sung

May you live the rest of your day...

GeneralRe: IDisposable over FormPowerShellSample_FormClosing
Jean-Paul Mikkers
2:56 28 Aug '08  
Hi Sung,

Thanks, I am glad you liked them. About your question: did you mean to implement IDisposable on PipelineExecutor? If so, I didn't use that method because it's possible to reuse the runspace (to run other scripts) after the PipelineExecutor finishes. In my example code there is just one runspace during the lifetime of the process, even though you can run multiple scripts on it.

If you meant "do it somewhere else in the Form", then you have to be careful: the PipelineExecutor depends on a working messagequeue to marshal the DataReady and DataEnd events from the powershell thread to the main form.. if you terminate the scripts too late in the form closing process (i.e. when there's no more window handle and messagequeue) then the powershell thread will crash.

Well I hope I answered your question, if not, let me know Smile

Best Regards,
Jean-Paul
GeneralRe: IDisposable over FormPowerShellSample_FormClosing
dance2die
4:46 28 Aug '08  
Sorry about leaving out important contexs for IDisposable question.
I meant to ask was about placing logic in FormPowerShellSample form's FormPowerShellSample_FormClosing to Dispose method as a code below shows.


private void FormPowerShellSample_FormClosing(object sender, FormClosingEventArgs e)
{
//CleanUpManagedResources();
}

private bool m_IsDisposed = false;

protected override void Dispose(bool disposing)
{
if (m_IsDisposed) return;

if (disposing && (components != null))
{
components.Dispose();
CleanUpManagedResources();
}
base.Dispose(disposing);

m_IsDisposed = true;
}

private void CleanUpManagedResources()
{
// stop any running scripts
StopScript();
// close the powershell runspace
runSpace.Close();
}


I am not sure if above code will leak memory or cause other abnormalities.

May you live the rest of your day...

GeneralRe: IDisposable over FormPowerShellSample_FormClosing
Jean-Paul Mikkers
5:00 28 Aug '08  
Aha thanks for the clarification. Well I suspect this will cause problems IF the script is still running and producing output. This is because the PipelineExecutor will try to pass the script output to the Form by way of the ISynchronizeInvoke interface, but the window handle that is needed to process the message is no longer available.

Closing the runspace there should be fine though.
GeneralRe: IDisposable over FormPowerShellSample_FormClosing
dance2die
8:20 28 Aug '08  
I am now stopping scripts within "FormClosing" event and Dispose simply closes runspace.
It seems to work great~.

Thank you for helpful feedbacks.

-- Sung

May you live the rest of your day...

GeneralWhere is Pipeline.Error?
spambuster
5:55 28 Sep '07  
For commands that raise an exception or other syntax error nothing is returned. What would be beneficial is to include handling of error messages as well.
GeneralRe: Where is Pipeline.Error?
juFo
5:43 30 Mar '08  
Yes how would you return errors?
GeneralRe: Where is Pipeline.Error?
Jean-Paul Mikkers
23:59 29 Aug '08  
You're absolutely right.. this was long overdue, but I finally managed to spare some time to update the article. It now shows how you can intercept errors, and how to handle script syntax errors.
QuestionOutput for external commands
joeyb2007
20:49 12 Jul '07  
Hi, this code works perfect for cmdlets but whenever i attempt to execute a powershell function i get no output. How do I get the following code to execute and return the output using your PipelineExecuter class?


$TFExecutablePath = "C:\Program Files\Microsoft Visual Studio 8\Common7\IDE"

function DeleteWorkspace([String]$Name, [String]$TeamFoundationServerUrl)
{
$ErrorActionPreference = "stop"
# delete the workspace
& $TFExecutablePath\.\tf.exe workspace /delete $Name /s:$TeamFoundationServerUrl /noprompt
# spit out the exit code
write-Host "ExitCode: $lastexitcode"
}

DeleteWorkspace "Test" "MyServer"


Since I'm running an external executable in this function I want to see the output from the call to it and the following statements are what I'm expecting at the moment.

TF14061: The workspace Test;domain\user does not exist.
ExitCode: 100

AnswerRe: Output for external commands
Jean-Paul Mikkers
12:57 14 Jul '07  
Hi joey,

If you host powershell scripts in your own programs, all output that you write using "write-host" gets chucked away. There are two solutions to your problem:

1. - supply the method RunspaceFactory.CreateRunspace() with your own PSHost implementation. If you want an example, just let me know and I'll mail it to you.

or

2. - replace that last "write-Host" by "return". This will cause the string to be pushed into the output pipeline so that PipelineExecuter can process it.

Regards,
Jean-Paul

GeneralRe: Output for external commands
Ragna Rok
15:40 29 Jan '08  
Try adding "| Out-String -stream" to the end of the DeleteWorkspace line
QuestionPipelineExecutor for items other than powershell
sides_dale
16:11 22 Apr '07  
I really like your PipelineExecutor method.Is there a way this can be implemented for items other than powershell scripts? This would be handy to have as a method to use for some of my other programs multi-threading is always a bear to me.
AnswerRe: PipelineExecutor for items other than powershell
Jean-Paul Mikkers
22:49 23 Apr '07  

Sure, I suppose it's possible to make a class that allows easy execution of a worker thread while still providing an easy way to communicate results to the user interface. Have you looked at the 'BackgroundWorker' class (in the components section of your toolbox)?


Last Updated 29 Aug 2008 | Advertise | Privacy | Terms of Use | Copyright © CodeProject, 1999-2010