Async Await and the Generated StateMachine

Introduction

Like everybody around, I went digging into .NET 4.5 to see what's new... but I didn't find any good step by step explanations about how the state machine actually works, so I used ilspy, and along with a good video from the creator, I got a good notion of what is what. (and you are welcome to have a peek too).

The State Machine

I read somewhere that you can think of it like this:

• Call "before" code
• Call await code as task
• Store "after" code in a continuation task

But this isn’t really the case.

What really happens is actually something like this:

• On compile time:
• A struct called StateMachine is generated
• Contains fields to save function local state
• A moveNext function is created which holds the entire code
• The code is fragmented by await calls into several cases (machine states)
• A calling code which creates and initializes this machine replaces our async function code.
• On Runtime: 
• A task is created to run the machine code:
• Local variables are “lifted” into the state machine as fields.
• Code is run until await
• A awaited function's task is run
• Machine state is set to next state so next code fragment will run on wakeup
• A wake-up event is scheduled
• MoveNext function returns (Thread is released to do other stuff (update UI))
• When wakeup call is issued by OS: 
• Thread which handles the await continuation is called
• CurrentSyncContext is used to pick the correct thread to run it on.
• This behavior can be changed by using: await task.ConfigureAwait(false);
• Next code segment is run since next state was set before yielding control
• Another await is scheduled, [etc.]

See it written in the code

Two async functions decoded

// AsyncAwait.Engine.Downloader
public static async Task<string> DownloadHtmlAsyncTask(string url)
{
    HttpClient httpClient = new HttpClient();
    Debug.WriteLine("before await");
    string result = await httpClient.GetStringAsync(url);
    Debug.WriteLine("after await");
    return result;
}

// AsyncAwait.ViewModel.MainWndViewModel
private async Task<string> DownloadWithUrlTrackingTaskAsync(string url)
{
    Debug.WriteLine("before await1");
    string Data = await DownloadHtmlAsyncTask(url);
    Debug.WriteLine("before await2");
    string Data1 = await DownloadHtmlAsyncTask(url);
    Debug.WriteLine("the end.");
    return Data;
}

The decompilation was done (using ilspy) with "decompile async methods" turned off.

First function

DownloadHtmlAsyncTask uses only one await call.

This is the calling code which initializes the state machine:

[DebuggerStepThrough, AsyncStateMachine(typeof(AsyncMethods.<DownloadHtmlAsyncTask>d__0))]
public static Task<string> DownloadHtmlAsyncTask(string url)
{
    AsyncMethods.<DownloadHtmlAsyncTask>d__0 <DownloadHtmlAsyncTask>d__;
    <DownloadHtmlAsyncTask>d__.url = url;
    <DownloadHtmlAsyncTask>d__.<>t__builder = AsyncTaskMethodBuilder<string>.Create();

    //set initial machine state to -1
    <DownloadHtmlAsyncTask>d__.<>1__state = -1;
    AsyncTaskMethodBuilder<string> <>t__builder = <DownloadHtmlAsyncTask>d__.<>t__builder;
    <>t__builder.Start<AsyncMethods.<DownloadHtmlAsyncTask>d__0>(ref <DownloadHtmlAsyncTask>d__);
    return <DownloadHtmlAsyncTask>d__.<>t__builder.get_Task();

}

The state machine definition: (my comments inline)

using ConsoleApplication1;
using System;
using System.Diagnostics;
using System.Net.Http;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
[CompilerGenerated]
[StructLayout(LayoutKind.Auto)]
private struct <DownloadHtmlAsyncTask>d__0 : IAsyncStateMachine
{
    //initial state is set to -1 by the machine's creation code
    public int <>1__state;
    public AsyncTaskMethodBuilder<string> <>t__builder;
    public string url;
    public HttpClient <httpClient>5__1;
    public string <result>5__2;
    private TaskAwaiter<string> <>u__$awaiter3;
    private object <>t__stack;
    void IAsyncStateMachine.MoveNext()
    {
        string result;
        try
        {
            int num = this.<>1__state;

            //if (!Stop)
            if (num != -3)
            {
                TaskAwaiter<string> taskAwaiter;
                //machine starts with num=-1 so we enter
                if (num != 0)
                {
                    //first (+ initial) state code, run code before await is invoked
                    this.<httpClient>5__1 = new HttpClient();
                    Debug.WriteLine("before await");
                    
                    //a task is invoked
                    taskAwaiter = this.<httpClient>5__1.GetStringAsync(this.url).GetAwaiter();
                    
                    //[performance] check if this task has completed already,
                    //if it did, skip scheduling and boxing
                    if (!taskAwaiter.get_IsCompleted())
                    {
                        this.<>1__state = 0;
                        this.<>u__$awaiter3 = taskAwaiter;
                        
                        //Schedules the state machine to proceed
                        //to the next action when the specified awaiter completes.
                        //Also: sending this state machine here will trigger it's boxing into heap.
                        this.<>t__builder.AwaitUnsafeOnCompleted<TaskAwaiter<string>, 
                          AsyncMethods.<DownloadHtmlAsyncTask>d__0>(ref taskAwaiter, ref this);
                        
                        //release cpu knowing our next step will be called by Framework.
                        return;
                    }
                }
                else
                {
                    //set awaiter to null
                    taskAwaiter = this.<>u__$awaiter3;
                    this.<>u__$awaiter3 = default(TaskAwaiter<string>);
                    
                    //set state to initial state (temporarily)
                    this.<>1__state = -1;
                }
                
                //second (+ final) state code (state=0): set result to member, printout
                string arg_A5_0 = taskAwaiter.GetResult();
                
                //set awaiter to null
                taskAwaiter = default(TaskAwaiter<string>);
                
                //set StateMachine's result field, and end code (print out)
                string text = arg_A5_0;
                this.<result>5__2 = text;
                Debug.WriteLine("after await");
                
                //set return task's result
                result = this.<result>5__2;
            }
        }
        //exception handling is done here
        catch (Exception exception)
        {
            //set machine state to final state
            this.<>1__state = -2;
            this.<>t__builder.SetException(exception);
            return;
        }
        this.<>1__state = -2;
        this.<>t__builder.SetResult(result);
    }
    [DebuggerHidden]
    void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine param0)
    {
        this.<>t__builder.SetStateMachine(param0);
    }
}

• State machine is created as a struct, which means it's on stack
• It is moved to heap inside the AwaitUnsafeOnCompleted function (deeper into the mechanism it is cast to IAsyncStateMachine which triggers the boxing). 
• This call may be skipped if the task we await ends before the taskAwaiter.IsCoplete is checked.

This function has only two code fragments (two states) since there is only one await dividing the code, it may be a standard scenario but not an interesting one.

What happens when the StateMachine has more states? this brings us to the second example...

Second function

DownloadHtmlAsyncTask - using two await calls:

The calling code:

[DebuggerStepThrough, AsyncStateMachine(typeof(AsyncMethods.<DownloadWithUrlTrackingTaskAsync>d__5))]
private Task<string> DownloadWithUrlTrackingTaskAsync(string url)
{
    AsyncMethods.<DownloadWithUrlTrackingTaskAsync>d__5 <DownloadWithUrlTrackingTaskAsync>d__;
    <DownloadWithUrlTrackingTaskAsync>d__.<>4__this = this;
    <DownloadWithUrlTrackingTaskAsync>d__.url = url;
    <DownloadWithUrlTrackingTaskAsync>d__.<>t__builder = AsyncTaskMethodBuilder<string>.Create();
    
    //set initial machine state to -1
    <DownloadWithUrlTrackingTaskAsync>d__.<>1__state = -1;
    AsyncTaskMethodBuilder<string> <>t__builder = 
      <DownloadWithUrlTrackingTaskAsync>d__.<>t__builder;
    <>t__builder.Start<AsyncMethods.<DownloadWithUrlTrackingTaskAsync>d__5>(ref <DownloadWithUrlTrackingTaskAsync>d__);
    return <DownloadWithUrlTrackingTaskAsync>d__.<>t__builder.get_Task();
}

The state machine definition: (read the comments in the code)

using ConsoleApplication1;
using System;
using System.Diagnostics;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
[CompilerGenerated]
[StructLayout(LayoutKind.Auto)]
private struct <DownloadWithUrlTrackingTaskAsync>d__5 : IAsyncStateMachine
{
    public int <>1__state;
    public AsyncTaskMethodBuilder<string> <>t__builder;
    public AsyncMethods <>4__this;
    public string url;
    public string <Data>5__6;
    public string <Data1>5__7;
    private TaskAwaiter<string> <>u__$awaiter8;
    private object <>t__stack;
    void IAsyncStateMachine.MoveNext()
    {
        string result;
        try
        {
            TaskAwaiter<string> taskAwaiter;
            //initialy state = -1, so we skip this at start
            switch (this.<>1__state)
            {
            case -3:
                //request machine to stop!
                //machine will go to end state and set result if one exists.
                goto IL_168;
            case 0:
                taskAwaiter = this.<>u__$awaiter8;
                this.<>u__$awaiter8 = default(TaskAwaiter<string>);
                //set state to initial state (temporarily)
                this.<>1__state = -1;
                goto IL_A1;
            case 1:
                taskAwaiter = this.<>u__$awaiter8;
                this.<>u__$awaiter8 = default(TaskAwaiter<string>);
                //set state to initial state (temporarily)
                this.<>1__state = -1;
                goto IL_121;
            }
            
            // first state code state=-1, 
                        //printout, and await, then return control to scheduler
            Debug.WriteLine("before await1");
            taskAwaiter = AsyncMethods.DownloadHtmlAsyncTask(this.url).GetAwaiter();
            if (!taskAwaiter.get_IsCompleted())
            {
                //set state to next step (0)
                this.<>1__state = 0;
                this.<>u__$awaiter8 = taskAwaiter;
                this.<>t__builder.AwaitUnsafeOnCompleted<TaskAwaiter<string>, 
                  AsyncMethods.<DownloadWithUrlTrackingTaskAsync>d__5>(ref taskAwaiter, ref this);
                return;
            }
            IL_A1:
            //second state code (state = 0), set the result of the first call to member, 
                        //printout, schedule next await, yield control
            string arg_B0_0 = taskAwaiter.GetResult();
            taskAwaiter = default(TaskAwaiter<string>);
            string text = arg_B0_0;
            this.<Data>5__6 = text;
            Debug.WriteLine("before await2");
            taskAwaiter = AsyncMethods.DownloadHtmlAsyncTask(this.url).GetAwaiter();
            if (!taskAwaiter.get_IsCompleted())
            {
                //set state to next step (1)
                this.<>1__state = 1;
                this.<>u__$awaiter8 = taskAwaiter;
                this.<>t__builder.AwaitUnsafeOnCompleted<TaskAwaiter<string>, 
                  AsyncMethods.<DownloadWithUrlTrackingTaskAsync>d__5>(ref taskAwaiter, ref this);
                return;
            }
            IL_121:
            //third state code (state = 1), 
                        //set the result of the first call to member, printout, 
                        //set the (function's) return task's result.
            string arg_130_0 = taskAwaiter.GetResult();
            taskAwaiter = default(TaskAwaiter<string>);
            text = arg_130_0;
            this.<Data1>5__7 = text;
            Debug.WriteLine("the end.");
            result = this.<Data>5__6;
        }
        catch (Exception exception)
        {
            //some exception handling: set end state (-2)
            this.<>1__state = -2;
            //set the exception in the builder
            this.<>t__builder.SetException(exception);
            return;
        }
        IL_168:
        //if no exception set end state and result
        this.<>1__state = -2;
        this.<>t__builder.SetResult(result);
    }
    [DebuggerHidden]
    void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine param0)
    {
        this.<>t__builder.SetStateMachine(param0);
    }
}

Points of Interest

• The overhead:
• Anyone can see that there are "several" extra code lines added.
• But according to Microsoft:
• About 40 operations are used to create async state machine (microseconds).
• This overhead is similar or less than the other older async mechanisms.
• This can be a problem only when calling many awaits in a tight loop.
• Besides that, with today's CPUs who will notice the overhead when it's run on a non-GUI thread? Or even better - on the HDD controller...
• Exception handling
• Task.WhenAll will throw any exception from any of the tasks it aggregates.
• This means that the first exception will be thrown out.
• The task.Exception property will contain all the exceptions inside an AggregateException (you can use Task.Wait() instead of await)