Azure Storage Blobs Service Working with Directories

Introduction

Windows Azure Blobs Service is combined from the following components:

1. Storage account
2. Containers
3. Blobs
4. Pages/Blocks

The components above have the following relationship: A storage account holds many containers , each container holds many blobs and each blob is combined from pages or blocks (depends on the type of Blob created).

Azure Blobs Service provides a well defined URI scheme: http://<account>.blob.core.windows.net/<container>/<blobname> For example : http://queststorage.blob.core.windows.net/images/landscape/banner.png

Resources in the Azure Storage Services (Blobs/Queues/Tables) are exposed via REST api’s. Well pulling the sleeves and creating a REST request isn't necessary ( though not to complicated) since Microsoft provides the Microsoft.WindowsAzure.StorageClient Namespace API in order to perform a large verity of operations against the blobs service like creating/deleting/downloading container/blobs etc.. (I will describe later a an overview of the core classes and some important methods to use).

If we look carefully at the blob URL above we can see that something is missing from our discussion:

Lets review the URL again: http://queststorage.blob.core.windows.net/images/landscape/banner.png

The queststorage is the storage account name , images is the blobs container but what is landscape/banner.png?

To first answer the question. The missing concept that the Blobs service expose is called directories. To be specific a directory is nothing but a prefix to the blob name in order to provide some virtual blob hierarchy inside the blobs service. This merely helps organize the blobs inside the container.

Using the code

Microsoft.WindowsAzure.StorageClient

Before going to the code lets view the core class inside and the design structure between them Microsoft.WindowsAzure.StorageClient

CloudBlob - is the base class for CloudBlobDirectory, CloudBlockBlob and CloudPageBlob

Those classes implements the IListBlobItem which contains 3 getters:

• CloudBlobContainer Container
• CloudBlobDirectory Parent
• Uri Uri.

Connecting to the Storage account

One more class that is very important is CloudBlobClient This class provides the client in order to access the blobs storage.

//
// Connecting to the storage account
// 
string storageName = "storageAccount";
string storageKey = "AvailableFromTheAzurePortal";
CloudStorageAccount storage = CloudStorageAccount.Parse(String.Format("DefaultEndpointsProtocol=https;AccountName={0};AccountKey={1}", storageName, storageKey));
CloudBlobClient cloudBlobClient = storage.CreateCloudBlobClient();

Working with Directories

Not that we know how to connect to the blob service lets find the relevant api that help us to work with directories.

First lets create a blob container in our storage account:

CloudBlobContainer container = cloudBlobClient.GetContainerReference("mycontainer"); 
container.CreateIfNotExist(); 

// Set public access level to the container.
container.SetPermissions(new BlobContainerPermissions()
{
    PublicAccess = BlobContainerPublicAccessType.Container
});

Now that we have container let populate it with a simple file.

Since we want to create a directory hierarchy in our storage we'll set the blob name to have a directories prefix separated by slashes. So the blob name will be dira/dirb/file

// upload a file to a blob in azure using directory name prefix
using (FileStream stream = new FileStream(@"c:\temp\" + fileName, FileMode.Open, FileAccess.Read))
{
    CloudBlockBlob blob = container.GetBlockBlobReference("dira/dirb/simpleText.txt");

    const int blockSize = 256 * 1024;
    byte[] blockData = new byte[blockSize];
    string blockID;
    List<string> blockIDs = new List<string>();
    int blockNumber = 0;
    int bytesRead = 0;

    long size = stream.Length;
    while (bytesRead < size)
    {
        int n = stream.Read(blockData, 0, blockSize);
        bytesRead += n;

        if (n == 0)
        {
            break;
        }

        blockID = Convert.ToBase64String(BitConverter.GetBytes(blockNumber));
        blockIDs.Add(blockID);
        using (MemoryStream memoryStream = new MemoryStream())
        {
            memoryStream.Write(blockData, 0, n);
            memoryStream.Position = 0;
            blob.PutBlock(blockID, memoryStream, null);
        }

        ++blockNumber;
    }
    stream.Close();

    blob.PutBlockList(blockIDs);
}

Next lets access the directories we created above.

Azure Storage client provides the following API in order the get a reference to the Cloud Directory.

CloudBlobDirectory dira = container.GetDirectoryReference("dira");

We can also get all the blobs inside that directory easily:

List<IListBlobItem> blobs = dira.ListBlobs().ToList();

Lets drill down to the sub-directory.

CloudBlobDirectory dirb =  dira.GetSubdirectory("dirb");

We can also go navigate up in directory tree:

var parent = dirb.Parent;

Finally we get all the blobs in the container and igonre the directory structure (flatten view) in the following way:

container.ListBlobs(new BlobRequestOptions() 
{ 
    UseFlatBlobListing = true 
});