Click here to Skip to main content
Click here to Skip to main content

Tagged as

Azure Storage Blobs Service Working with Directories

, 15 Jan 2012 CPOL
Rate this:
Please Sign up or sign in to vote.
Directories manipulation in Blobs Service

Introduction

Windows Azure Blobs Service is combined from the following components:

  1. Storage account
  2. Containers
  3. Blobs
  4. Pages/Blocks

The components above have the following relationship:
A storage account holds many containers , each container holds many blobs and
each blob is combined from pages or blocks (depends on the type of Blob created).

Azure Blobs Service provides a well defined URI scheme: http://<account>.blob.core.windows.net/<container>/<blobname>
For example : http://queststorage.blob.core.windows.net/images/landscape/banner.png

Resources in the Azure Storage Services (Blobs/Queues/Tables) are exposed via REST api’s.
Well pulling the sleeves and creating a REST request isn't necessary ( though not to complicated) since Microsoft provides the Microsoft.WindowsAzure.StorageClient Namespace API in order to perform a large verity of operations against the blobs service like creating/deleting/downloading container/blobs etc..
(I will describe later a an overview of the core classes and some important methods to use).

If we look carefully at the blob URL above we can see that something is missing from our discussion:

Lets review the URL again:
http://queststorage.blob.core.windows.net/images/landscape/banner.png

The queststorage is the storage account name , images is the blobs container but what is landscape/banner.png?

To first answer the question.
The missing concept that the Blobs service expose is called directories.
To be specific a directory is nothing but a prefix to the blob name in order to provide some virtual blob hierarchy inside the blobs service.
This merely helps organize the blobs inside the container.

Using the code

Microsoft.WindowsAzure.StorageClient

Before going to the code lets view the core class inside and the design structure between them
Microsoft.WindowsAzure.StorageClient

CloudBlob - is the base class for CloudBlobDirectory, CloudBlockBlob and CloudPageBlob

Those classes implements the IListBlobItem which contains 3 getters:

  • CloudBlobContainer Container
  • CloudBlobDirectory Parent
  • Uri Uri.
we can use those getters in order to traverse between the components (for example to get the parent directory of a blob etc..).

Connecting to the Storage account

One more class that is very important is CloudBlobClient
This class provides the client in order to access the blobs storage.

//
// Connecting to the storage account
// 
string storageName = "storageAccount";
string storageKey = "AvailableFromTheAzurePortal";
CloudStorageAccount storage = CloudStorageAccount.Parse(String.Format("DefaultEndpointsProtocol=https;AccountName={0};AccountKey={1}", storageName, storageKey));
CloudBlobClient cloudBlobClient = storage.CreateCloudBlobClient(); 

Working with Directories

Not that we know how to connect to the blob service lets find the relevant api that help us to work with directories.

First lets create a blob container in our storage account:

CloudBlobContainer container = cloudBlobClient.GetContainerReference("mycontainer"); 
container.CreateIfNotExist(); 

// Set public access level to the container.
container.SetPermissions(new BlobContainerPermissions()
{
    PublicAccess = BlobContainerPublicAccessType.Container
});   

Now that we have container let populate it with a simple file.

Since we want to create a directory hierarchy in our storage we'll set the blob name to have a directories prefix separated by slashes.
So the blob name will be dira/dirb/file

// upload a file to a blob in azure using directory name prefix
using (FileStream stream = new FileStream(@"c:\temp\" + fileName, FileMode.Open, FileAccess.Read))
{
    CloudBlockBlob blob = container.GetBlockBlobReference("dira/dirb/simpleText.txt");

    const int blockSize = 256 * 1024;
    byte[] blockData = new byte[blockSize];
    string blockID;
    List<string> blockIDs = new List<string>();
    int blockNumber = 0;
    int bytesRead = 0;

    long size = stream.Length;
    while (bytesRead < size)
    {
        int n = stream.Read(blockData, 0, blockSize);
        bytesRead += n;

        if (n == 0)
        {
            break;
        }

        blockID = Convert.ToBase64String(BitConverter.GetBytes(blockNumber));
        blockIDs.Add(blockID);
        using (MemoryStream memoryStream = new MemoryStream())
        {
            memoryStream.Write(blockData, 0, n);
            memoryStream.Position = 0;
            blob.PutBlock(blockID, memoryStream, null);
        }

        ++blockNumber;
    }
    stream.Close();

    blob.PutBlockList(blockIDs);
}  

Next lets access the directories we created above.

Azure Storage client provides the following API in order the get a reference to the Cloud Directory.

CloudBlobDirectory dira = container.GetDirectoryReference("dira"); 

We can also get all the blobs inside that directory easily:

List<IListBlobItem> blobs = dira.ListBlobs().ToList(); 

Lets drill down to the sub-directory.

CloudBlobDirectory dirb =  dira.GetSubdirectory("dirb");  

We can also go navigate up in directory tree:

var parent = dirb.Parent; 

Finally we get all the blobs in the container and igonre the directory structure (flatten view) in the following way:

container.ListBlobs(new BlobRequestOptions() 
{ 
    UseFlatBlobListing = true 
});    

Points of Interest

In Quest Software we develop a very handy tool called Cloud Storage Manager for Azure .

You can browse and manipulate data in the Azure storage account very intuitively.

The following display the directory structure we've created in the examples above using our tool:

blobs.PNG

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Shay Yannay
Software Developer Quest Software
Israel Israel
Shay Yannay is a Senior Software Developer and Windows Azure Domain Expert.
He is experienced with designing and developing highly scalable, distributed, 24x7 availability complex system. Shay also specializes in performance management & diagnostics of multi-tier applications.
He is passionate about the cloud technologies and trends, specifically with Microsoft Azure.
He currently works for Dell as an Azure Specialist.
 
Shay holds a B.Sc in Communication Systems Engineering from the Ben-Gurion university.
 
Personal Blog: http://shayyannay.wordpress.com
Follow on   Twitter

Comments and Discussions

 
QuestionUpload Multiple images to azure and retrieve using MVC PinmemberMember 917112829-Aug-13 0:40 
Generalthanks Pinmemberplayhomess10-Mar-13 22:38 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.1411023.1 | Last Updated 15 Jan 2012
Article Copyright 2011 by Shay Yannay
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid