Click here to Skip to main content
12,696,493 members (32,048 online)
Click here to Skip to main content
Add your own
alternative version


5 bookmarked

CQRS on Windows Azure - The command side

, 3 May 2016 CPOL
Rate this:
Please Sign up or sign in to vote.
An outline of how a paired azure queue and azure table can be used for the command side of a CQRS application


This article shows a way of using Windows Azure storage -specifically the Queue and the Table storage - to implement the command side of a Command Query Responsibility Segregation architecture.  It is implemented in VB.Net but there is nothing in it preventing a very quick conversion to C# if that is your language of choice.
It does not cover the query side of the architecture nor the related concepts of Event Sourcing which I hope to get to as the application this code comes from is built out.


At its simplest CQRS is an architecture pattern that separates commands (doing) from queries (asking). What this means in practice is that the command and query side do not need to share a common model and this vertical shearing layer is often matched with a vertical shearing layer between the definition (of a command or query) and the implementation (in a command handler or a query handler).

In my experience, CQRS is very useful when you are developing an application with a distributed team to a very short timescale as it reduces the risk of "model contention" which can occur if everyone is updating the model at the same time.  It is also useful for financial scenarios where having an audit trail of all the changes that have occurred in an application is required.  


(If you are not familiar with CQRS can I recommend you start with this excellent article for an overview of the how and why) 


Defining the command 

Every type of action you can perform is considered its own command and each command has a definition - covering both what the command is and what additional parameters are needed. You need a distinct class for each type of command your system uses which defined both what the command is, and what payload is needed in order to execute it.
For example if you have a bank account based system you might have a command OpenAccount that has parameters that cover the data needed to open a bank account such as the account owner, holding branch, denomination currency and so on whereas a command DepositFunds would require an account number and an amount. You could also have a command that does not require any parameters at all.

In order that a given class can be semantically identified as being a command it derives from a standard interface ICommandDefinition. This also adds two mandatory properties - an unique identifier by which every instance of a command can be identified and a human-readable name by which the command is known.
(Many implementations do not use a human readable command name - you can use it or not)

''' <summary>
''' Interface defining what is required of a class that defines a command to be executed
''' </summary>
''' <remarks>
''' Commands can have named parameters that have meaning to the command handler
''' </remarks>
Public Interface ICommandDefinition

    ''' <summary>
    ''' Unique identifier of this command instance
    ''' </summary>
    ''' <remarks>
    ''' This allows commands to be queued and the handler to know if a command has been executed or not (for example as an Event Source)
    ''' </remarks>
    ReadOnly Property InstanceIdentifier As Guid

    ''' <summary>
    ''' The unique name of the command being executed
    ''' </summary>
    ReadOnly Property CommandName As String

End Interface

At this point we can go ahead and define concrete classes for each command - for example our OpenAccount command definition could look like:

Public Class OpenAccountCommandDefinition
    Implements ICommandDefinition

    Private m_instanceid As Guid = Guid.NewGuid

    Public ReadOnly Property InstanceIdentifier As Guid Implements ICommandDefinition.InstanceIdentifier
            Return m_instanceid
        End Get
    End Property

    Public Overrides ReadOnly Property CommandName As String
          Return "Open a new account"
       End Get
    End Property

    'Payload for this command
    Public Property BranchIdentifier As String

    Public Property AccountOwner As String

    Public Property DenominationCurrency As String

End Class

However, in order to pass the command definition across to the command handler I am using windows azure storage - specifically a queue and a matching table. The queue has the command type and unique identifier appended on to it and the matching table is used to hold any additional payload that the command requires.
The maximum size of a message on an Azure queue is 64kb but it is possible that a command would require more payload than that so this dual approach is used

Dispatching the command

The first step in putting the payload in an Azure table is to identify the payload properties of the command to save them off. This can either be done using reflection or by having a class the represents a command parameter and using a dictionary of them to back the payload properties.

When a command definition has been created it then needs to be dispatched. The command dispatcher is responsible for the transfer to the command handlers. In the case of the Azure dispatcher this is a two step process- first save the payload to a commands table, then add a message to the queue. The (partial) code snippet to do this is:-

Dim applicationStorageAccount As CloudStorageAccount
Dim applicationQueueClient As CloudQueueClient
Dim applicationTableClient As CloudTableClient

Public Sub Send(command As ICommandDefinition) Implements ICommandDispatcher.Send

    ' 1) Save the command data in a commands table for the handler to use
    If (applicationTableClient IsNot Nothing) Then
        Dim commandTable As CloudTable = applicationTableClient.GetTableReference("commandsparameterstable")
        'Table may not exists yet, so create it if it doesn't
        Dim cmdRecord As New CommandTableEntity(command)
        '\\ Insert or update the record
        Dim insertOperation As TableOperation = TableOperation.InsertOrReplace(cmdRecord)
        Dim insertResult = commandTable.Execute(insertOperation)
    End If

    ' 2) Queue the command to execute
    If (applicationQueueClient IsNot Nothing) Then
        Dim queue As CloudQueue = applicationQueueClient.GetQueueReference("commandsqueue")
        ' Queue may not yet exist so create it is it doesn't
        Dim msg As CloudQueueMessage = New CloudQueueMessage(command.GetType.Name & "::" & command.InstanceIdentifier.ToString())
        If (msg IsNot Nothing) Then
        End If
    End If

End Sub

It is important to follow the naming standards for queues and tables - all lower case and no punctuation.

In order to save a command record to an azure table it must either be put in a class that inherits from TableEntity or, if you are doing manipulations behind the scenes then you would need a class that inherits from ITableEntity.
I wasn't able to find many examples of inheriting from ITableEntity so mine is as follows:

''' <summary>
 ''' A class for storing a command and its parameters in an azure table store
 ''' </summary>
 ''' <remarks>
 ''' The command type and the instance identifier are the partition and row keys to allow any
 ''' command handler easily to find its command parameter data
 ''' </remarks>
 Public Class CommandTableEntity
     Implements ITableEntity

     Private m_parameters As Dictionary(Of String, CommandParameter) = New Dictionary(Of String, CommandParameter)

     ''' <remarks>
     ''' Set this value to '*' in order to blindly overwrite an entity as part of an update operation.
     ''' </remarks>
     Public Property ETag As String Implements ITableEntity.ETag

     ''' <summary>
     ''' The property that defines what table partition this command will be stored in
     ''' </summary>
     Public Property PartitionKey As String Implements ITableEntity.PartitionKey

     ''' <summary>
     ''' The property that defines what row this command record will be stored in
     ''' </summary>
     Public Property RowKey As String Implements ITableEntity.RowKey

     Public Property Timestamp As DateTimeOffset Implements ITableEntity.Timestamp

     ''' <summary>
     ''' Has this command been marked as processed
     ''' </summary>
     Public Property Processed As Boolean

     ''' <summary>
     ''' Has this command been marked as in error
     ''' </summary>
     Public Property InError As Boolean

     ''' <summary>
     ''' What handler processed this command
     ''' </summary>
     Public Property ProcessedBy As String

     Public Sub ReadEntity(properties As IDictionary(Of String, EntityProperty), operationContext As OperationContext) Implements ITableEntity.ReadEntity

         For Each propertyPair In properties
             If (propertyPair.Key.Contains("__")) Then
                 'This is a property in the form name[index]...
                 Dim fixedPropertyName As String = propertyPair.Key.Replace("__", "[").Trim() & "]"
                 Dim parameterName As String = CommandParameter.GetParameterName(fixedPropertyName)
                 Dim parameterIndex As Integer = CommandParameter.GetParameterIndex(fixedPropertyName)
                 'Get the command payload from the string
                 Dim parameterPayload As String = propertyPair.Value.StringValue
                 If (Not String.IsNullOrWhiteSpace(parameterPayload)) Then
                     m_parameters.Add(propertyPair.Key, CommandParameter.Create(parameterPayload))
                     '\\ Add a parameter that has no payload/value
                     m_parameters.Add(propertyPair.Key, CommandParameter.Create(parameterName, parameterIndex))
                 End If
                 'Named property not persist "ETag", "Timestamp", "Rowkey" or "PartitionKey"

                 ' Additional properties for a command
                 If propertyPair.Key.Equals("Processed", StringComparison.OrdinalIgnoreCase) Then
                     If (propertyPair.Value.BooleanValue.HasValue) Then
                         Me.Processed = propertyPair.Value.BooleanValue.Value
                     End If
                 End If

                 If propertyPair.Key.Equals("InError", StringComparison.OrdinalIgnoreCase) Then
                     If (propertyPair.Value.BooleanValue.HasValue) Then
                         Me.InError = propertyPair.Value.BooleanValue.Value
                     End If
                 End If

                 If propertyPair.Key.Equals("ProcessedBy", StringComparison.OrdinalIgnoreCase) Then
                     Me.ProcessedBy = propertyPair.Value.StringValue
                 End If

             End If

     End Sub

     Public Function WriteEntity(operationContext As OperationContext) As IDictionary(Of String, EntityProperty) Implements ITableEntity.WriteEntity

         Dim properties As New Dictionary(Of String, EntityProperty)
         'Skip the ITableEntity properties "ETag", "Timestamp", "Rowkey" or "PartitionKey"

         'Add all the command parameters...
         For Each param As CommandParameter In m_parameters.Values
             Dim fixedPropertyName As String = CommandParameter.GetParameterKey(param).Replace("[", "__").Replace("]", "")
             properties.Add(fixedPropertyName, New EntityProperty(param.ToString))

         'Add the other properties
         properties.Add("Processed", New EntityProperty(Me.Processed))
         properties.Add("InError", New EntityProperty(Me.InError))
         properties.Add("ProcessedBy", New EntityProperty(Me.ProcessedBy))

         Return properties

     End Function
 End Class

Next step - getting the command off the queue and processing it - for the next article...

Querying the command

As we have given each command instance it's own unique identifier in InstanceIdentifier, we can create a user interface component to query the state of the command queue and table so as to perform a "How is my command getting on" type of query. 

This solves one of the biggest areas of confusion in CQRS - how does the user get feedback that their command has executed if it is totally asynchronous.  The solution is for them to ask - either by explicit user interaction or by some polling mechanism.

This also means that any systems that don't require status feedback (such as automated sensor feeds or data pipelines) can use exactly the same command handling functions as are used by an interactive front end.

Points of Interest

If you are doing any work with Windows Azure Storage I recommend downloading the Azure Storage Explorer project from CodePlex.

Tips I have learnt so far:-
1) When storing data in windows azure table storage, put extra care into deciding what the partition key property is - I started out with too high level an object (client) which meant my tables only had a small number of partitions which makes data access slower.
2) It costs a (very small) amount of money to poll a windows storage queue even if it is empty therefore a way of throttling the polling rate up or down depending on demand is a very good idea


 v1 - 2014-01-22 Rewritten to emphasise why over how - removed the code dump aspect

 v2 - 2014-01-19 Outline of the classes involved

 v3 - 2016-03-05 Added thoughts on querying the command


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

Duncan Edwards Jones
Software Developer (Senior)
Ireland Ireland
C# / SQL Server developer
Microsoft MVP 2006, 2007
Visual Basic .NET

You may also be interested in...


Comments and Discussions

Question[My vote of 1] noooo dammit! Pin
bryce19-Jan-14 15:31
memberbryce19-Jan-14 15:31 
AnswerRe: [My vote of 1] noooo dammit! Pin
Duncan Edwards Jones19-Jan-14 21:44
memberDuncan Edwards Jones19-Jan-14 21:44 
AnswerRe: [My vote of 1] noooo dammit! Pin
Duncan Edwards Jones22-Jan-14 8:43
memberDuncan Edwards Jones22-Jan-14 8:43 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web02 | 2.8.170118.1 | Last Updated 3 May 2016
Article Copyright 2014 by Duncan Edwards Jones
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid