Click here to Skip to main content
Click here to Skip to main content

Windows Azure Storage Extensions

, 11 Aug 2013
Rate this:
Please Sign up or sign in to vote.
.NET library aimed for managing and querying entities from Windows Azure Storage. Contains LINQ to Azure Table provider.

Introduction

Windows Azure Storage Extensions is a .NET library aimed for managing and querying entities from Windows Azure Storage. It's built on top of the Windows Azure SDK v2.0, provides asynchronous interfaces (Task-based Asynchronous Pattern) and LINQ to Azure Table queries via TableSet context by using POCO entities. 

Table of Contents

Windows Azure Storage Extensions

Background

Currently Windows Azure SDK v1.x with LINQ to Azure Table provider marked as obsolete. Its successor Windows Azure SDK v2.0 was written from scratch and has few breaking changes which includes new querying mechanism. It's implementation has following gaps.

Querying by using TableQuery 

Now if we don't want to use legacy WCF Data Services we should construct our queries to Azure Tables by using ugly code like that:

string filter = TableQuery.GenerateFilterCondition(
    "RowKey", QueryComparisons.GreaterThanOrEqual, "5");
TableQuery<TableEntity> query = new TableQuery<TableEntity>().Where(filter).Take(5);
var entities = myTable.ExecuteQuery(query);

Requirements to derive from TableEntity in POCO objects

For using POCO entities we should use TableEntity base class: 

public class SampleEntity : TableEntity
{
    public int SampleProperty { get; set; }  
}

Except that it grant us ITableEntity interface with PartitionKey, RowKey, Timestamp, and ETag properties it uses slow Reflection for entities serialization / deserialization. 

Low Abstraction Level

For me as a software engineer Azure Storage Library v2.0 looks too low level abstraction of Azure Tables REST API. For instance when I want to use DTO entities I must remember that PartitionKey is a user identifier, actually it's not meaningful and we should use some kind of entity mappers. Querying also is a too complicated task.

Windows Azure Storage Extensions

Windows Azure Storage Extensions library offering following features. 

POCO 

Entity's properties and fields should be marked by one or both of PartitionKey and RowKey attributes for defining composite table key. Also can be used Timestamp, ETag, Property and Ignore attributes.

Attribute Description Constraint 
PartitionKey Defines a partition key property. Only for string properties.
RowKey  Defines a row key property. Only for string properties.
ETag Defines an etag property. Only for string properties.
Timestamp Used for receiving table entity timestamp. Only for DateTime or DateTimeOffset properties.
Property Used for defining custom table property name via Name property.
Ignore Used for skipping properties from serialization / deserialization.

Table Entities Management

Generic TableSet context provides both synchronous & asynchronous (TAP) methods for managing entities:

  • Synchronous: Add, AddOrUpdate, Update, and Remove
  • Asynchronous: AddAsync, AddOrUpdateAsync, UpdateAsync, and RemoveAsync.

To avoid restrictions of group operations in Azure Storage all entities sorted by partition keys and merged into groups by 100 entities. Execution of requests with such batch operations can be configured via TableSet's ExecutionMode property. Allowed values:

  • Sequential
  • Parallel

Default ExecutionMode is Sequential.

LINQ to Azure Tables 

TableSet context implements IQueryable interface for using LINQ Expressions. Provider supports next synchronous LINQ methods:

  • First
  • FirstOrDefault
  • Single
  • SingleOrDefault
  • Take
  • Where

To utilize filtering capabilities of string properties it supports following methods: 

Also you can use Contains method. In this case query statement for each collection's item will be joined by using OData or operator.

Note: For creating a custom queries you should take a look at next article Mixing LINQ Providers and LINQ to Objects.

Asynchronous LINQ Queries

In addition TableSet can be used for asynchronous queries powered by LINQ extensions (TAP) in EF 6 Async style. Available methods:

  • FirstAsync
  • FirstOrDefaultAsync
  • SingleAsync
  • SingleOrDefaultAsync
  • TakeAsync
  • ToListAsync

LINQ Projections

LINQ Projections supported with a limitation - projection class should be a reference type.

Task-based Extension Methods

Library contains TAP-based extensions for a following Windows Azure SDK classes:

  • CloudBlobClient
  • CloudBlobContainer
  • CloudTableClient
  • CloudTable

To use it just add Async postfix to synchronous method name like that:

blobs = cloudBlobContainer.ListBlobs();
blobs = await cloudBlobContainer.ListBlobsAsync();

Task Cancellation

All of TAP-based methods accepts optional CancellationToken parameter for Task Cancellation.

Download

Via NuGet

To install library by using Windows Azure Storage Extensions nuget package execute following command:

PM> Install-Package WindowsAzure.StorageExtensions 

Via Git

To get the source code of the library via git just type:

git clone git://github.com/dtretyakov/WindowsAzure.git
cd ./WindowsAzure 

Example

You can download SPA example powered by ASP.NET MVC4 & Web API. It uses Azure Table Storage as a persistence, so before you run that example start Azure Storage Emulator or configure proper connection string in the Web.config.

Example App

One of the interesting things in this example is a Get method in the IssuesController. At runtime SPA sends an OData queries to WebAPI IssuesController which by leveraging Queryable attribute transforms into LINQ queries to the TableSet context which in turn translates LINQ expressions into OData filters for Azure Storage Table service. 

Code Samples

Defining a new class:

public sealed class Country
{
    [PartitionKey]
    public string Continent { get; set; }
    [RowKey]
    public string Name { get; set; }
    public long Population { get; set; }
    public double Area { get; set; }
    public DateTime Formed { get; set; }
}

Creating a new table context:

var storageAccount = CloudStorageAccount.DevelopmentStorageAccount;
var tableClient = storageAccount.CreateCloudTableClient();

var countryTable = new TableSet<Country>(tableClient);

Adding a new entities:

var resultSync = countryTable.Add(country);
var resultAsync = await countryTable.AddAsync(country);

Updating an entities:

resultSync.Area += 333333;
resultSync = countryTable.Update(resultSync);
 
resultAsync.Population *= 2;
resultAsync = await countryTable.UpdateAsync(resultAsync);

Removing an entities: 

countryTable.Remove(resultSync);
await countryTable.RemoveAsync(resultAsync);

Querying entities:

var query = countryTable.Where(
    p => p.Formed > new DateTime(1950, 1, 1) &&
        (p.PresidentsCount < 10 || p.Population < 10000000 && p.IsExists));
 
var result = query.ToList();
result = await query.ToListAsync();

or using string filtering methods:

var countryStartingWithF = await countryTable.FirstAsync(
    p => p.Name.CompareTo("F") >= 0 && p.Name.CompareTo("G") < 0);

Using LINQ projections:

var projection = from country in countryTable
    where country.Area > 400000
    select new { country.Continent, country.Name };

var entities = projection.ToList();
entities = await projection.ToListAsync();

Using Contains method in the LINQ query:

var countryNames = new List<string> { "Germany", "Finland" };
var countries = countryTable.Where(p => countryNames.Contains(p.Name)).ToList();

Points of Interest

Source code of the library are available at the github.com/dtretyakov/windowsazure.  Core components of the Windows Azure Storage Extensions are following.

Table Context

ITableSet

TableSet<T> class is an abstraction over Azure Storage Table. It implements IQueryable<T> and ITableSet<T> interfaces. Internals in actions it performs single and batch requests to the Azure Storage Tables by using CloudTable class and uses POCO object serialization by leveraging TableEntityConverter<T>.

Table Entity Converter

TableEntityConverter<T> implements ITableEntityConverter<T> interface. It uses compiled Expression Trees for fastest access to POCO fields and properties. Internally it converts POCO objects into DynamicTableEntity and back. Performance comparison results in ms of TableEntityConverter for POCO, TableEntity and new operator are following:

Conversion 10M 50M
DynamicTableEntity to Object 8501 43,690
DynamicTableEntity to POCO 17,826 89,907
DynamicTableEntity to TableEntity 54,158 276,421
Object to DynamicTableEntity 10,693 51,485
POCO to DynamicTableEntity 15,528 78,368
TableEntity to DynamicTableEntity 41,471 218,935

Tests can be found in: EntityConverterTests.cs

Table Query Provider

TableQueryProvider<T> implements IQueryProvider and IAsyncQueryProvider interfaces. It provides translation of LINQ Expressions into Azure Storage Table OData filters and able to execute it synchronously and asynchronously.

Task-based Extension Methods 

Windows Azure SDK v2.0 provides us APM-based interfaces and ICancellableAsyncResult for operation cancellation. .NET 4.0 and higher uses other TAP-based asynchronous programming model. Thus Windows Azure Storage Extensions provides APM to TAP wrappers for most of  Windows Azure SDK classes. Library internally uses these extensions for executing asynchronous requests to Azure Storage Tables.

Feedback 

Your suggestions and comments are very welcome at the GitHub project page

History

  • 0.1.0 - December 16, 2012: Alpha version.
  • 0.7.0 - April 12, 2013: First stable version.
  • 0.7.2 - April 22, 2013: Added LINQ projections, entities partitioning.
  • 0.7.3 - April 27, 2013: Serialization performance improvements, ITableSet's methods provides IEnumerable again.
  • 0.7.4 - May 10, 2013: Added Contains method; performance optimization of the LINQ to OData translator.
  • 0.7.6 - May 20, 2013: Performance optimizations and code stabilization.
  • 0.7.7 - August 10, 2013: Code stabilization. 

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Dmitry Tretyakov
Software Developer (Senior)
Russian Federation Russian Federation
Software engineer with an application development experience.
 
MCSD: Web Applications
MCSD: Windows Store Apps Using C#

Comments and Discussions

 
QuestionGreat help PinmemberDuncan Edwards Jones16-Jan-14 0:37 
GeneralMy vote of 5 PinmemberIInjac31-Aug-13 14:53 
GeneralMy vote of 5 PinmemberKim Togo18-Aug-13 23:16 
QuestionVery cool PinmemberGreg D Engle13-Jun-13 15:47 
QuestionThank you! Very useful. PinmemberMike Gurfing7-May-13 1:26 
AnswerRe: Thank you! Very useful. PinmemberDmitry Tretyakov7-May-13 6:44 
SuggestionCongratulations PinmemberLuiz Bicalho6-May-13 7:23 
AnswerRe: Congratulations PinmemberDmitry Tretyakov6-May-13 8:51 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140721.1 | Last Updated 12 Aug 2013
Article Copyright 2013 by Dmitry Tretyakov
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid