Windows Azure Storage Extensions

• Download example web app - 814.2 KB

Introduction

Windows Azure Storage Extensions is a .NET library aimed for managing and querying entities from Windows Azure Storage. It's built on top of the Windows Azure SDK v2.0, provides asynchronous interfaces (Task-based Asynchronous Pattern) and LINQ to Azure Table queries via TableSet context by using POCO entities. 

Table of Contents

• Background
• Features
• Download
• Example
• Code Samples
• Points of Interest

Background

Currently Windows Azure SDK v1.x with LINQ to Azure Table provider marked as obsolete. Its successor Windows Azure SDK v2.0 was written from scratch and has few breaking changes which includes new querying mechanism. It's implementation has following gaps.

Querying by using TableQuery 

Now if we don't want to use legacy WCF Data Services we should construct our queries to Azure Tables by using ugly code like that:

string filter = TableQuery.GenerateFilterCondition(
    "RowKey", QueryComparisons.GreaterThanOrEqual, "5");
TableQuery<TableEntity> query = new TableQuery<TableEntity>().Where(filter).Take(5);
var entities = myTable.ExecuteQuery(query);

Requirements to derive from TableEntity in POCO objects

For using POCO entities we should use TableEntity base class: 

public class SampleEntity : TableEntity
{
    public int SampleProperty { get; set; }  
}

Except that it grant us ITableEntity interface with PartitionKey, RowKey, Timestamp, and ETag properties it uses slow Reflection for entities serialization / deserialization. 

Low Abstraction Level

For me as a software engineer Azure Storage Library v2.0 looks too low level abstraction of Azure Tables REST API. For instance when I want to use DTO entities I must remember that PartitionKey is a user identifier, actually it's not meaningful and we should use some kind of entity mappers. Querying also is a too complicated task.

Windows Azure Storage Extensions

Windows Azure Storage Extensions library offering following features. 

POCO 

Entity's properties and fields should be marked by one or both of PartitionKey and RowKey attributes for defining composite table key. Also can be used Timestamp, ETag, Property and Ignore attributes.

Attribute	Description	Constraint
PartitionKey	Defines a partition key property.	Only for string properties.
RowKey	Defines a row key property.	Only for string properties.
ETag	Defines an etag property.	Only for string properties.
Timestamp	Used for receiving table entity timestamp.	Only for DateTime or DateTimeOffset properties.
Property	Used for defining custom table property name via Name property.
Ignore	Used for skipping properties from serialization / deserialization.

Table Entities Management

Generic TableSet context provides both synchronous & asynchronous (TAP) methods for managing entities:

• Synchronous: Add, AddOrUpdate, Update, and Remove. 
• Asynchronous: AddAsync, AddOrUpdateAsync, UpdateAsync, and RemoveAsync.

To avoid restrictions of group operations in Azure Storage all entities sorted by partition keys and merged into groups by 100 entities. Execution of requests with such batch operations can be configured via TableSet's ExecutionMode property. Allowed values:

• Sequential
• Parallel

Default ExecutionMode is Sequential.

LINQ to Azure Tables 

TableSet context implements IQueryable interface for using LINQ Expressions. Provider supports next synchronous LINQ methods:

• First
• FirstOrDefault
• Single
• SingleOrDefault
• Take
• Where

To utilize filtering capabilities of string properties it supports following methods: 

• Compare
• CompareTo
• CompareOrdinal

Also you can use Contains method. In this case query statement for each collection's item will be joined by using OData or operator.

Note: For creating a custom queries you should take a look at next article Mixing LINQ Providers and LINQ to Objects.

Asynchronous LINQ Queries

In addition TableSet can be used for asynchronous queries powered by LINQ extensions (TAP) in EF 6 Async style. Available methods:

• FirstAsync
• FirstOrDefaultAsync
• SingleAsync
• SingleOrDefaultAsync
• TakeAsync
• ToListAsync

LINQ Projections

LINQ Projections supported with a limitation - projection class should be a reference type.

Task-based Extension Methods

Library contains TAP-based extensions for a following Windows Azure SDK classes:

• CloudBlobClient
• CloudBlobContainer
• CloudTableClient
• CloudTable

To use it just add Async postfix to synchronous method name like that:

blobs = cloudBlobContainer.ListBlobs();
blobs = await cloudBlobContainer.ListBlobsAsync();

Task Cancellation

All of TAP-based methods accepts optional CancellationToken parameter for Task Cancellation.

Download

Via NuGet

To install library by using Windows Azure Storage Extensions nuget package execute following command:

PM> Install-Package WindowsAzure.StorageExtensions 

Via Git

To get the source code of the library via git just type:

git clone git://github.com/dtretyakov/WindowsAzure.git
cd ./WindowsAzure 

Example

You can download SPA example powered by ASP.NET MVC4 & Web API. It uses Azure Table Storage as a persistence, so before you run that example start Azure Storage Emulator or configure proper connection string in the Web.config.

One of the interesting things in this example is a Get method in the IssuesController. At runtime SPA sends an OData queries to WebAPI IssuesController which by leveraging Queryable attribute transforms into LINQ queries to the TableSet context which in turn translates LINQ expressions into OData filters for Azure Storage Table service. 

Code Samples

Defining a new class:

public sealed class Country
{
    [PartitionKey]
    public string Continent { get; set; }
    [RowKey]
    public string Name { get; set; }
    public long Population { get; set; }
    public double Area { get; set; }
    public DateTime Formed { get; set; }
}

Creating a new table context:

var storageAccount = CloudStorageAccount.DevelopmentStorageAccount;
var tableClient = storageAccount.CreateCloudTableClient();

var countryTable = new TableSet<Country>(tableClient);

Adding a new entities:

var resultSync = countryTable.Add(country);
var resultAsync = await countryTable.AddAsync(country);

Updating an entities:

resultSync.Area += 333333;
resultSync = countryTable.Update(resultSync);
 
resultAsync.Population *= 2;
resultAsync = await countryTable.UpdateAsync(resultAsync);

Removing an entities: 

countryTable.Remove(resultSync);
await countryTable.RemoveAsync(resultAsync);

Querying entities:

var query = countryTable.Where(
    p => p.Formed > new DateTime(1950, 1, 1) &&
        (p.PresidentsCount < 10 || p.Population < 10000000 && p.IsExists));
 
var result = query.ToList();
result = await query.ToListAsync();

or using string filtering methods:

var countryStartingWithF = await countryTable.FirstAsync(
    p => p.Name.CompareTo("F") >= 0 && p.Name.CompareTo("G") < 0);

Using LINQ projections:

var projection = from country in countryTable
    where country.Area > 400000
    select new { country.Continent, country.Name };

var entities = projection.ToList();
entities = await projection.ToListAsync();

Using Contains method in the LINQ query:

var countryNames = new List<string> { "Germany", "Finland" };
var countries = countryTable.Where(p => countryNames.Contains(p.Name)).ToList();

Points of Interest

Source code of the library are available at the github.com/dtretyakov/windowsazure.  Core components of the Windows Azure Storage Extensions are following.

Table Context

TableSet<T> class is an abstraction over Azure Storage Table. It implements IQueryable<T> and ITableSet<T> interfaces. Internals in actions it performs single and batch requests to the Azure Storage Tables by using CloudTable class and uses POCO object serialization by leveraging TableEntityConverter<T>.

Table Entity Converter

TableEntityConverter<T> implements ITableEntityConverter<T> interface. It uses compiled Expression Trees for fastest access to POCO fields and properties. Internally it converts POCO objects into DynamicTableEntity and back. Performance comparison results in ms of TableEntityConverter for POCO, TableEntity and new operator are following:

Conversion	10M	50M
DynamicTableEntity to Object	8501	43,690
DynamicTableEntity to POCO	17,826	89,907
DynamicTableEntity to TableEntity	54,158	276,421
Object to DynamicTableEntity	10,693	51,485
POCO to DynamicTableEntity	15,528	78,368
TableEntity to DynamicTableEntity	41,471	218,935

Tests can be found in: EntityConverterTests.cs

Table Query Provider

TableQueryProvider<T> implements IQueryProvider and IAsyncQueryProvider interfaces. It provides translation of LINQ Expressions into Azure Storage Table OData filters and able to execute it synchronously and asynchronously.

Task-based Extension Methods 

Windows Azure SDK v2.0 provides us APM-based interfaces and ICancellableAsyncResult for operation cancellation. .NET 4.0 and higher uses other TAP-based asynchronous programming model. Thus Windows Azure Storage Extensions provides APM to TAP wrappers for most of  Windows Azure SDK classes. Library internally uses these extensions for executing asynchronous requests to Azure Storage Tables.

Feedback 

Your suggestions and comments are very welcome at the GitHub project page. 

History

• 0.1.0 - December 16, 2012: Alpha version.
• 0.7.0 - April 12, 2013: First stable version.
• 0.7.2 - April 22, 2013: Added LINQ projections, entities partitioning.
• 0.7.3 - April 27, 2013: Serialization performance improvements, ITableSet's methods provides IEnumerable again.
• 0.7.4 - May 10, 2013: Added Contains method; performance optimization of the LINQ to OData translator.
• 0.7.6 - May 20, 2013: Performance optimizations and code stabilization.
• 0.7.7 - August 10, 2013: Code stabilization.