I already wrote the article Yield Return Could Be Better and I must say that
async/await could be better if a stack-saving mechanism was implemented to do real cooperative threading.
I am not saying that the
async/await is a bad thing, but it could be added without compiler changes (enabling any .NET compiler to use it) and maybe adding the keywords to make its usage explicit.
Different from the other time, I will not only talk about the advantages, I will provide a sample implementation of a stacksaver and show its benefits.
Understanding the Async/Await Pair
The async/await was planned for the .NET 5 but it is already available in the 4.5 CTP. Its promise is to make asynchronous code easier to write, which it indeed does.
But my problem with it is: Why people want to use the asynchronous pattern to begin with?
The main reason is: To keep the UI responsive.
We can already maintain the UI responsive using secondary threads. So, what's the real difference?
Well, let's see this pseudo-code:
using(var reader = ExecuteReader())
Very simple, a reader is created and while there are records, they are added to a listbox.
But imagine that it has 60 records, and that each ReadRecord takes one second to complete. If you put that code in the Click of a Button, your UI will freeze for an entire minute.
If you put that code in a secondary thread you will have problems when adding the items to the listbox, so you will need to use something like
listbox.Dispatcher.Invoke to really update the listbox.
With the new
await keyword your method will need to be marked as
async and you will need to change the while line, like this:
And your UI will be responsible.
Your UI became responsible by a simple call to
And what's that
Well, here is where the complexity really lives. The
await is, in fact, registering a continuation and then allowing the actual method to finish immediatelly (in the case of a Button Click, the thread is free to further process UI messages). Everything that cames after the
await will be stored in another method, and any data used before and after the
await keyword will live in another class created by the compiler and passed as a parameter to that continuation.
Then there is the implementation of
ReadRecordAsync. This one may be considered the hardest part, as it may use some kind of real asynchronous completion (like IO completion ports of the operating system) or it will still use a secondary thread, like a
If it still uses secondary threads, you may wonder how is it going to be faster than a normal secondary thread.
Well... it is not going to be faster, it may be a little slower as it by default needs to send a message back to the UI thread when the process is completed. But if you are going to update the UI, you will already need to do that.
Some speed advantage may reside on the fact that the actual thread may already start something else (instead of waiting doing nothing) and also on the
ThreadPool usually used by the
Tasks, which forbid too many concurrent tasks. Some tasks need to end so new
Tasks can start. With normal threads we may risk having too many threads trying to run at once (much more than the real processor count), when it will be faster to let some threads simple wait to start.
Noticing the obvious
Independent of the benefits of the
ThreadPool and the ease of use of the
async keyword, did you notice that when you put an
await in a method the actual thread is free to do another job (like processing further UI messages)?
And that at some point such
await will receive a result and continue? With that you can very easily start 5 different jobs. Each one, at the end, will continue running on the same thread (probably the UI).
It is not hard to see those jobs as "slim" threads. As a
Job, they start, they "block" awaiting, and they continue. The real thread can do other things in the "blocking" part, but the same already happens with the CPU when a real thread enters a blocking state (the CPU continues doing other things while the thread is blocked).
Such Jobs don't necessarely have priorities, they run as a simple queue in their manager thread but everytime they finish or enter in a "wait state" they allow the next job to run.
So, they will all run in the same real thread, and one Job must await or finish to allow others to run. That's cooperative threading.
It Could Be Better
I said at the beginning that it could be better so, how?
Well, real cooperative threads will do the same as the await keyword, but without the await keyword, without returning a Task and consequently making the code more prepared to future changes.
You may think that code using await is prepared for future changes, but do you remember my pseudo-code?
using(var reader = ExecuteReader())
Imagine that you update it to use the await keyword. At this moment, only the ReadRecord method is asynchronous, so the code ends-up like this:
using(var reader = ExecuteReader())
But in the future the
ExecuteReader method (which is almost instantaneous today) may take 5 seconds to respond. What do I do then?
I should create an
ExecuteReaderAsync, that will return a
Task and should replace all the calls to
ExecuteReader() by an
await ExecuteReaderAsync(). That will be a giant breaking change.
Wouldn't it be better if the
ExecuteReader itself was able to tell "I am going to sit and wait, so let another job run in my place"?
Pausing and Resuming a Job
Here is where all the problems are concentrated and here is the reason
await keyword exists. Well, I think people at Microsoft got so fascinated that they could change the compiler to manage a secondary callstacks using objects and delegates (effectively creating the continuation) that they forgot they can create a full new callstack and replace it.
If you don't know what the callstack is, you may have already seen it in the debugger window. It keeps track of all methods that are actually executing and all variables. If method A calls method B, which then calls method C, it will have the exact position in method C, the position it will be when C returns and also the position to return to A when B returns.
A continuation is the hard version of this. In fact, simple continuing with another method is easy, the problem is creating a
try/catch block in method A and putting a continuation to B that is still in the same
try/catch. In fact the compiler will create an entire
try/catch in method A and in method B, both executing the same code in the
catch (probably with an additional method to be reutilize by the catch code).
If instead of managing a "secondary callstack" in a continuation they created a completely new callstack and replaced the thread callstack by the new and, at wait points, restored the original callstack, it will be much simpler as all the code that uses the callstack will continue to use it. No additional methods or different control flows to deal with
Such alternative callstack is what I called a
StackSaver in the other article but my original idea was misleading. It does not need to save and restore part ot the callstack. It is a completely separated callstack that can be be used in place of the normal callstack (and will restore the original callstack in waits or as its last action). It will be a "single pointer" change to do all the job (or even a single CPU register change).
Good Theory, but It Will Not Work
The .Net team did a lot of changes to support the "compiler magic" to make the async work and I tell that if we can simple create new callstacks we can have the same benefits with an even easier to use and more maintenable code, and that all we need is to be able to switch from one callstack to another.
That looks too simple and maybe you think that I am missing something, even if you don't know what, and so you believe it will not work.
Well, that's why I created my simulation of a
StackSaver to prove that it works.
My simulation uses full threads to store the callstack, after all there is no way to switch from one callstack to another at the moment. But this is a simulation, and it will prove my point.
Even being full threads, I am not simple letting them run in parallel as that will have all the problems related to concurrency (and will be normal threading). The
StackSaver class is fully synchronized to its main thread, so only one runs at a time.
This will give the sensation of:
- Calling the
StackSaver.Execute to start executing the other callstack "in the actual thread";
- When the action running in the
StackSaver ends or calls
StackSaver.YieldReturn, the control goes back to the original callstack.
The only big difference of my
StackSaver is that anything that uses the
Thread identify (like WPF) will notice that it is another thread. So it is not a real replacement but works for my simulation purposes and already allows to create a
yield return replacement without any compiler tricks.
You din't saw wrong I am not committing an error, by default the
StackSaver allows for a
yield return replacement, not for an
Doing the Async/Await replacement with the StackSaver
To use the
StackSaver as an
async/await replacement we must have a thread that deals with one or more
StackSavers. I am calling the class that creates such thread as
It runs like an eternal loop. If there are no jobs, it waits (real thread waiting, no job waiting). If there are one or more
Jobs, it dequeues a
Job and makes it run. As soon as it returns, (by a
yield return or by finishing) and the original caller regains execution, it checks if it should put the Job again in the queue (as the last one) or not.
The only problem then is to wait for something. When the Job request a "blocking" operation it must create a
CooperativeWaitEvent, will set-up how the async part of the job really works (maybe using the
ThreadPool, maybe using IO completion ports) will mark itself as waiting and will
The main callstack, after seeing the Job is waiting, will not put it in the execution queue again. But when the real operation ends and "Sets" the wait event, it will requeue the job.
It is simple as that and here is the entire code of the CooperativeJobManager:
public sealed class CooperativeJobManager:
private readonly HashSet<CooperativeJob> _allTasks = new HashSet<CooperativeJob>();
internal readonly Queue<CooperativeJob> _queuedTasks = new Queue<CooperativeJob>();
internal bool _waiting;
private bool _wasDisposed;
var thread = new Thread(_RunAll);
public void Dispose()
_wasDisposed = true;
public bool WasDisposed
private void _RunAll()
CooperativeJob task = null;
if (_queuedTasks.Count == 0)
if (task == null)
if (_wasDisposed && _allTasks.Count == 0)
_waiting = true;
while (_queuedTasks.Count == 0);
if (task != null)
if (_queuedTasks.Count != 0)
_waiting = false;
task = _queuedTasks.Dequeue();
CooperativeJob._current = task;
if (!task._Continue() || task._waiting)
task = null;
public CooperativeJob Run(Action action)
if (action == null)
throw new ArgumentNullException("action");
var result = new CooperativeJob(this);
var stackSaver = new StackSaver(() => _Run(result, action));
result._stackSaver = stackSaver;
private void _Run(CooperativeJob task, Action action)
CooperativeJob._current = task;
CooperativeJob._current = null;
With it you can call Run passing an Action and that action will start as a
If the action never calls a
CooperativeJob.YieldReturn or some cooperative blocking call, it will effectively execute the action directly. It the action does some kind of yield or cooperative wait, then another job can run in its thread.
Now imagine this in your old Windows Forms application. At each UI event you call the CooperativeJobManager.Run to execute the real code. In those codes, any operation that may block (like acessing databases, files or even Sleeps) allow another job to run. And that's all, you have full asynchronous code that does not has the complication of multi-threading and really looks like synchronous code.
The source for download is done in .NET 3.5 and I am sure it may work even under the .NET 1.0 (maybe requiring some changes).
The real missing thing is the StackSaver class which, as I already told, is using real threads in this implementation, so it is more useful for demonstration purposes only.
Advantages of the Cooperative Threading Over the Async/Await Done by the Compiler
- Will be available to any .Net compiler if it is in a class like the one presented here.
- You will not cause a breaking change if one method that today does not "block" starts to "block" in the future.
- You will not have an easier continuation style, because you can simple avoid it. In any place you need a continuation, create a new Job that may "block" without affecting your thread responsiveness.
- The callstack will be used normally, avoiding a cpu register used to store a reference to the "state" and another one already used by the callstack, which should make things a little faster.
- By having the callstack there, it will be easier to debug.
Advantages of the Async/Await Done by the Compiler over the Cooperative Threading
I can only see one. It is explicit, so users can't say they faced an asynchronous problem when they did synchronous code.
But that can be easily solved in the cooperative threading by flags that will effectively tell the
CooperativeJob that it cannot "block", raising an exception if a "blocking" call is done. It is certainly easier to make an area as "must not run other jobs here" than to have to await 10 times to do 10 different reads or writes.
Blocking versus "Blocking"
From my writing you may notice that a "blocking" call is not the same as a blocking call.
A "blocking" call blocks the actual job but let's the thread run freely. A real blocking call blocks the thread and, when it returns, it continues running the same job.
Surely it may be problematic if we have a framework full of blocking and "blocking" calls. But Microsoft is already reinventing everything with Metro (and even Silverlight has a network API that is asynchronous only).
So, why not replace all thread-blocking calls with job-blocking calls and make programming async software as easy as normal blocking software?
Did you like the idea?
Then ask microsoft to add real cooperative threading, through a stack-saver by clicking in this link and then voting for it.
I only did a very simple sample to show the difference of a real thread-blocking call versus a job-blocking call.
I am surely missing better samples and maybe I will add them later. Do not let the simplicity of the sample kill the real potential of the callstack "switching" mechanism, which can make better versions of asynchronous code,
yield return and also opens a lot of new scenarios for cooperative programming, making it easier to write more isolated code, that can both scale and be prepared for future improvement without breaking changes.
POLAR - The First Implementation of a StackSaver
I am finally presenting the first version of a
StackSaver for the .Net itself (even if it is a simulation) but this is not the first time I show a working version of the concept. I already presented it working in my POLAR language.
The language is still an hibrid between compilation and interpretation, but it uses the
stacksaver as a real callstack replacement and it will be relatively easy to implement asynchronous calls to it using the
Job concept instead of the
await keyword. I don't have a date for it as I am doing too many things at the time (like still adapting to a new country), but I can guarantee that it could be capable of working with such
Jobs without even knowing how to deal with secondary threads.
Coroutines and Fibers
When I started writing this article I didn't really know what coroutines where and I had no idea what a fiber was.
Well, at this moment I am really considering renaming my StackSaver class to Coroutine, as that is what it is really providing. And Fibers are the OS resource that allows to save the callstack and jump to another one and is the resource needed to create coroutines.
I did try to implement the StackSaver class using Fibers through P/Invoke but unfortunately the unmanaged Fibers don't really work in .NET. I really think that it is related to garbage collection, after all when searching for root objects the .NET will not see the "alternative callstacks" created by unmanaged fibers and will collect objects that are still alive, but unseen.
Either way, at this moment I will keep the name StackSaver and "Jobs", as this is similar to task but does not causes trouble with the