Click here to Skip to main content
Click here to Skip to main content
Go to top

Async/Await Could Be Better

, 30 Mar 2012
Rate this:
Please Sign up or sign in to vote.
This article will explain how the async/await pair really works and why it could be better if real cooperative threading was used instead
This is an old version of the currently published article.

Background

I already wrote the article Yield Return Could Be Better and I must say that async/await could be better if a stack-saving mechanism was implemented to do real cooperative threading.

I am not saying that the async/await is a bad thing, but it could be added without compiler changes (enabling any .NET compiler to use it) and maybe adding the keywords to make its usage explicit.

Different from the other time, I will not only talk about the advantages, I will provide a sample implementation of a stacksaver and show its benefits.

Understanding the Async/Await Pair

The async/await was planned for the .NET 5 but it is already available in the 4.5 CTP. Its promise is to make asynchronous code easier to write, which it indeed does.

But my problem with it is: Why people want to use the asynchronous pattern to begin with?

The main reason is: To keep the UI responsive.

We can already maintain the UI responsive using secondary threads. So, what's the real difference?

Well, let's see this pseudo-code:

using(var reader = ExecuteReader())
  while(reader.ReadRecord())
    listbox.Items.Add(reader.Current)

Very simple, a reader is created and while there are records, they are added to a listbox.

But imagine that it has 60 records, and that each ReadRecord takes one second to complete. If you put that code in the Click of a Button, your UI will freeze for an entire minute.

If you put that code in a secondary thread you will have problems when adding the items to the listbox, so you will need to use something like listbox.Dispatcher.Invoke to really update the listbox.

With the new await keyword your method will need to be marked as async and you will need to change the while line, like this:

while(await reader.ReadRecordAsync())

And your UI will be responsible.

That's magic!

Your UI became responsible by a simple call to await?

And what's that ReadRecordAsync?

Well, here is where the complexity really lives. The await is, in fact, registering a continuation and then allowing the actual method to finish immediatelly (in the case of a Button Click, the thread is free to further process UI messages). Everything that cames after the await will be stored in another method, and any data used before and after the await keyword will live in another class created by the compiler and passed as a parameter to that continuation.

Then there is the implementation of ReadRecordAsync. This one may be considered the hardest part, as it may use some kind of real asynchronous completion (like IO completion ports of the operating system) or it will still use a secondary thread, like a ThreadPool thread.

Secondary Threads

If it still uses secondary threads, you may wonder how is it going to be faster than a normal secondary thread.

Well... it is not going to be faster, it may be a little slower as it by default needs to send a message back to the UI thread when the process is completed. But if you are going to update the UI, you will already need to do that.

Some speed advantage may reside on the fact that the actual thread may already start something else (instead of waiting doing nothing) and also on the ThreadPool usually used by the Tasks, which forbid too many concurrent tasks. Some tasks need to end so new Tasks can start. With normal threads we may risk having too many threads trying to run at once (much more than the real processor count), when it will be faster to let some threads simple wait to start.

Noticing the obvious

Independent of the benefits of the ThreadPool and the ease of use of the async keyword, did you notice that when you put an await in a method the actual thread is free to do another job (like processing further UI messages)?

And that at some point such await will receive a result and continue? With that you can very easily start 5 different jobs. Each one, at the end, will continue running on the same thread (probably the UI).

It is not hard to see those jobs as "slim" threads. As a Job, they start, they "block" awaiting, and they continue. The real thread can do other things in the "blocking" part, but the same already happens with the CPU when a real thread enters a blocking state (the CPU continues doing other things while the thread is blocked).

Such Jobs don't necessarely have priorities, they run as a simple queue in their manager thread but everytime they finish or enter in a "wait state" they allow the next job to run.

So, they will all run in the same real thread, and one Job must await or finish to allow others to run. That's cooperative threading.

It Could Be Better

I said at the beginning that it could be better so, how?

Well, real cooperative threads will do the same as the await keyword, but without the await keyword, without returning a Task and consequently making the code more prepared to future changes.

You may think that code using await is prepared for future changes, but do you remember my pseudo-code?

using(var reader = ExecuteReader())
  while(reader.ReadRecord())
    listbox.Items.Add(reader.Current)

Imagine that you update it to use the await keyword. At this moment, only the ReadRecord method is asynchronous, so the code ends-up like this:

using(var reader = ExecuteReader())
  while(await reader.ReadRecordAsync())
    listbox.Items.Add(reader.Current)

But in the future the ExecuteReader method (which is almost instantaneous today) may take 5 seconds to respond. What do I do then?

I should create an ExecuteReaderAsync, that will return a Task and should replace all the calls to ExecuteReader() by an await ExecuteReaderAsync(). That will be a giant breaking change.

Wouldn't it be better if the ExecuteReader itself was able to tell "I am going to sit and wait, so let another job run in my place"?

Pausing and Resuming a Job

Here is where all the problems are concentrated and here is the reason await keyword exists. Well, I think people at Microsoft got so fascinated that they could change the compiler to manage a secondary callstacks using objects and delegates (effectively creating the continuation) that they forgot they can create a full new callstack and replace it.

If you don't know what the callstack is, you may have already seen it in the debugger window. It keeps track of all methods that are actually executing and all variables. If method A calls method B, which then calls method C, it will have the exact position in method C, the position it will be when C returns and also the position to return to A when B returns.

A continuation is the hard version of this. In fact, simple continuing with another method is easy, the problem is creating a try/catch block in method A and putting a continuation to B that is still in the same try/catch. In fact the compiler will create an entire try/catch in method A and in method B, both executing the same code in the catch (probably with an additional method to be reutilize by the catch code).

If instead of managing a "secondary callstack" in a continuation they created a completely new callstack and replaced the thread callstack by the new and, at wait points, restored the original callstack, it will be much simpler as all the code that uses the callstack will continue to use it. No additional methods or different control flows to deal with try/catches

Such alternative callstack is what I called a StackSaver in the other article but my original idea was misleading. It does not need to save and restore part ot the callstack. It is a completely separated callstack that can be be used in place of the normal callstack (and will restore the original callstack in waits or as its last action). It will be a "single pointer" change to do all the job (or even a single CPU register change).

Good Theory, but It Will Not Work

The .Net team did a lot of changes to support the "compiler magic" to make the async work and I tell that if we can simple create new callstacks we can have the same benefits with an even easier to use and more maintenable code, and that all we need is to be able to switch from one callstack to another.

That looks too simple and maybe you think that I am missing something, even if you don't know what, and so you believe it will not work.

Well, that's why I created my simulation of a StackSaver to prove that it works.

My simulation uses full threads to store the callstack, after all there is no way to switch from one callstack to another at the moment. But this is a simulation, and it will prove my point.

Even being full threads, I am not simple letting them run in parallel as that will have all the problems related to concurrency (and will be normal threading). The StackSaver class is fully synchronized to its main thread, so only one runs at a time.

This will give the sensation of:

  • Calling the StackSaver.Execute to start executing the other callstack "in the actual thread";
  • When the action running in the StackSaver ends or calls StackSaver.YieldReturn, the control goes back to the original callstack.

The only big difference of my StackSaver is that anything that uses the Thread identify (like WPF) will notice that it is another thread. So it is not a real replacement but works for my simulation purposes and already allows to create a yield return replacement without any compiler tricks.

You din't saw wrong I am not committing an error, by default the StackSaver allows for a yield return replacement, not for an async/await replacement.

Doing the Async/Await replacement with the StackSaver

To use the StackSaver as an async/await replacement we must have a thread that deals with one or more StackSavers. I am calling the class that creates such thread as CooperativeJobManager.

It runs like an eternal loop. If there are no jobs, it waits (real thread waiting, no job waiting). If there are one or more Jobs, it dequeues a Job and makes it run. As soon as it returns, (by a yield return or by finishing) and the original caller regains execution, it checks if it should put the Job again in the queue (as the last one) or not.

The only problem then is to wait for something. When the Job request a "blocking" operation it must create a CooperativeWaitEvent, will set-up how the async part of the job really works (maybe using the ThreadPool, maybe using IO completion ports) will mark itself as waiting and will yield return.

The main callstack, after seeing the Job is waiting, will not put it in the execution queue again. But when the real operation ends and "Sets" the wait event, it will requeue the job.

It is simple as that and here is the entire code of the CooperativeJobManager:

using System;
using System.Collections.Generic;
using System.Threading;

namespace Pfz.Threading.Cooperative
{
  public sealed class CooperativeJobManager:
    IDisposable
  {
    private readonly HashSet<CooperativeJob> _allTasks = new HashSet<CooperativeJob>();
    internal readonly Queue<CooperativeJob> _queuedTasks = new Queue<CooperativeJob>();
    internal bool _waiting;
    private bool _wasDisposed;

    public CooperativeJobManager()
    {
      // The real implementation uses my UnlimitedThreadPool class.
      // I removed such class in this version to give a smaller download
      // and make the right classes easier to find.
      var thread = new Thread(_RunAll);
      thread.Start();
    }
    public void Dispose()
    {
      lock(_queuedTasks)
      {
        _wasDisposed = true;

        if (_waiting)
          Monitor.Pulse(_queuedTasks);
      }
    }

    public bool WasDisposed
    {
      get
      {
        return _wasDisposed;
      }
    }

    private void _RunAll()
    {
      CooperativeJob task = null;
      //try
      //{
        while(true)
        {
          lock(_queuedTasks)
          {
            if (_queuedTasks.Count == 0)
            {
              if (task == null)
              {
                do
                {
                  if (_wasDisposed && _allTasks.Count == 0)
                    return;

                  _waiting = true;
                  Monitor.Wait(_queuedTasks);
                }
                while (_queuedTasks.Count == 0);
              }
            }
            else
            {
              if (task != null)
                _queuedTasks.Enqueue(task);
            }

            if (_queuedTasks.Count != 0)
            {
              _waiting = false;
              task = _queuedTasks.Dequeue();
            }
          }

          CooperativeJob._current = task;
          if (!task._Continue() || task._waiting)
            task = null;
        }
      //} will only work with real stacksavers.
      //finally
      //{
      //  CooperativeTask._current = null;
      //}
    }

    public CooperativeJob Run(Action action)
    {
      if (action == null)
        throw new ArgumentNullException("action");

      var result = new CooperativeJob(this);
      var stackSaver = new StackSaver(() => _Run(result, action));
      result._stackSaver = stackSaver;
      lock(_queuedTasks)
      {
        _allTasks.Add(result);
        _queuedTasks.Enqueue(result);

        if (_waiting)
          Monitor.Pulse(_queuedTasks);
      }

      return result;
    }
    private void _Run(CooperativeJob task, Action action)
    {
      try
      {
        CooperativeJob._current = task;
        action();
      }
      finally
      {
        CooperativeJob._current = null;

        lock(_allTasks)
          _allTasks.Remove(task);
      }
    }
  }
}

With it you can call Run passing an Action and that action will start as a CooperativeJob.

If the action never calls a CooperativeJob.YieldReturn or some cooperative blocking call, it will effectively execute the action directly. It the action does some kind of yield or cooperative wait, then another job can run in its thread.

Now imagine this in your old Windows Forms application. At each UI event you call the CooperativeJobManager.Run to execute the real code. In those codes, any operation that may block (like acessing databases, files or even Sleeps) allow another job to run. And that's all, you have full asynchronous code that does not has the complication of multi-threading and really looks like synchronous code.

The source for download is done in .NET 3.5 and I am sure it may work even under the .NET 1.0 (maybe requiring some changes).

The real missing thing is the StackSaver class which, as I already told, is using real threads in this implementation, so it is more useful for demonstration purposes only.

Advantages of the Cooperative Threading Over the Async/Await Done by the Compiler

  • Will be available to any .Net compiler if it is in a class like the one presented here.
  • You will not cause a breaking change if one method that today does not "block" starts to "block" in the future.
  • You will not have an easier continuation style, because you can simple avoid it. In any place you need a continuation, create a new Job that may "block" without affecting your thread responsiveness.
  • The callstack will be used normally, avoiding a cpu register used to store a reference to the "state" and another one already used by the callstack, which should make things a little faster.
  • By having the callstack there, it will be easier to debug.

Advantages of the Async/Await Done by the Compiler over the Cooperative Threading

I can only see one. It is explicit, so users can't say they faced an asynchronous problem when they did synchronous code.

But that can be easily solved in the cooperative threading by flags that will effectively tell the CooperativeJob that it cannot "block", raising an exception if a "blocking" call is done. It is certainly easier to make an area as "must not run other jobs here" than to have to await 10 times to do 10 different reads or writes.

Blocking versus "Blocking"

From my writing you may notice that a "blocking" call is not the same as a blocking call.

A "blocking" call blocks the actual job but let's the thread run freely. A real blocking call blocks the thread and, when it returns, it continues running the same job.

Surely it may be problematic if we have a framework full of blocking and "blocking" calls. But Microsoft is already reinventing everything with Metro (and even Silverlight has a network API that is asynchronous only).

So, why not replace all thread-blocking calls with job-blocking calls and make programming async software as easy as normal blocking software?

Did you like the idea?

Then ask microsoft to add real cooperative threading, through a stack-saver by clicking in this link and then voting for it.

The Sample

I only did a very simple sample to show the difference of a real thread-blocking call versus a job-blocking call.

I am surely missing better samples and maybe I will add them later. Do not let the simplicity of the sample kill the real potential of the callstack "switching" mechanism, which can make better versions of asynchronous code, yield return and also opens a lot of new scenarios for cooperative programming, making it easier to write more isolated code, that can both scale and be prepared for future improvement without breaking changes.

POLAR - The First Implementation of a StackSaver

I am finally presenting the first version of a StackSaver for the .Net itself (even if it is a simulation) but this is not the first time I show a working version of the concept. I already presented it working in my POLAR language.

The language is still an hibrid between compilation and interpretation, but it uses the stacksaver as a real callstack replacement and it will be relatively easy to implement asynchronous calls to it using the Job concept instead of the await keyword. I don't have a date for it as I am doing too many things at the time (like still adapting to a new country), but I can guarantee that it could be capable of working with such Jobs without even knowing how to deal with secondary threads.

Coroutines and Fibers

When I started writing this article I didn't really know what coroutines where and I had no idea what a fiber was.

Well, at this moment I am really considering renaming my StackSaver class to Coroutine, as that is what it is really providing. And Fibers are the OS resource that allows to save the callstack and jump to another one and is the resource needed to create coroutines.

I did try to implement the StackSaver class using Fibers through P/Invoke but unfortunately the unmanaged Fibers don't really work in .NET. I really think that it is related to garbage collection, after all when searching for root objects the .NET will not see the "alternative callstacks" created by unmanaged fibers and will collect objects that are still alive, but unseen.

Either way, at this moment I will keep the name StackSaver and "Jobs", as this is similar to task but does not causes trouble with the Task class.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Paulo Zemek
Architect
Canada Canada
I started to program computers when I was 11 years old, as a hobbist, programming in AMOS Basic and Blitz Basic for Amiga.
At 12 I had my first try with assembler, but it was too difficult at the time. Then, in the same year, I learned C and, after learning C, I was finally able to learn assembler (for Motorola 680x0).
Not sure, but probably between 12 and 13, I started to learn C++. I always programmed "in an object oriented way", but using function pointers instead of virtual methods.
 
At 15 I started to learn Pascal at school and to use Delphi. At 16 I started my first internship (using Delphi). At 18 I started to work professionally using C++ and since then I've developed my programming skills as a professional developer in C++ and C#, generally creating libraries that help other developers do they work easier, faster and with less errors.
 
Want more info or simply want to contact me?
Take a look at: http://paulozemek.azurewebsites.net/
Or e-mail me at: paulozemek@outlook.com
 
Codeproject MVP 2012
Microsoft MVP 2013

Comments and Discussions


Discussions posted for the Published version of this article. Posting a message here will take you to the publicly available article in order to continue your conversation in public.
 
GeneralJust wanted to say... Pinmemberkevininstructor12-Jun-14 11:45 
GeneralRe: Just wanted to say... PinpremiumPaulo Zemek12-Jun-14 12:13 
QuestionWOW - WOW - WOW !!! I just had to use async/await now and find it so stupid !!! PinmemberEric Ouellet11-Feb-14 10:24 
AnswerRe: WOW - WOW - WOW !!! I just had to use async/await now and find it so stupid !!! PinprofessionalPaulo Zemek11-Feb-14 10:38 
QuestionYour explanation could be better PinmemberDaniel Grunwald15-Apr-12 5:09 
AnswerRe: Your explanation could be better PinmvpPaulo Zemek15-Apr-12 9:22 
GeneralRe: Your explanation could be better PinmemberQwertie10-Jul-12 15:34 
GeneralRe: Your explanation could be better [modified] PinmemberQwertie10-Jul-12 15:18 
GeneralRe: Your explanation could be better PinmvpPaulo Zemek10-Jul-12 16:16 
GeneralRe: Your explanation could be better PinmemberQwertie11-Jul-12 5:09 
SuggestionHow the async/await pair really works PinmemberEugene Sadovoi10-Apr-12 8:55 
GeneralRe: How the async/await pair really works PinmvpPaulo Zemek10-Apr-12 9:26 
GeneralMy vote of 3 PinmemberEugene Sadovoi9-Apr-12 9:07 
GeneralRe: My vote of 3 PinmvpPaulo Zemek9-Apr-12 10:11 
SuggestionRe: My vote of 3 PinmemberEugene Sadovoi10-Apr-12 4:48 
GeneralRe: My vote of 3 PinmvpPaulo Zemek10-Apr-12 5:08 
GeneralRe: My vote of 3 PinmemberEugene Sadovoi10-Apr-12 6:07 
GeneralRe: My vote of 3 PinmvpPaulo Zemek10-Apr-12 6:47 
GeneralRe: My vote of 3 PinmemberEugene Sadovoi10-Apr-12 8:38 
GeneralRe: My vote of 3 PinmvpPaulo Zemek10-Apr-12 5:20 
GeneralRe: My vote of 3 PinmemberEugene Sadovoi10-Apr-12 6:10 
GeneralRe: My vote of 3 PinmvpPaulo Zemek10-Apr-12 6:53 
GeneralRe: My vote of 3 PinmemberEugene Sadovoi10-Apr-12 8:46 
GeneralRe: My vote of 3 PinmvpPaulo Zemek10-Apr-12 9:09 
GeneralRe: My vote of 3 PinmvpPaulo Zemek9-Apr-12 10:25 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web01 | 2.8.140905.1 | Last Updated 30 Mar 2012
Article Copyright 2012 by Paulo Zemek
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid