blog.smarx.com

Steve Marx's blog about cloud development

Sample Code for New Windows Azure Blob Features

[UPDATE 6/5/2009] This post is just about the new functionality in blob storage.  See the next blog post (“Sample Code for Batch Transactions in Windows Azure Tables”) to learn about the new functionality in table storage.


I’m a little bit late on this, but as promised on Thursday on the Windows Azure blog, I’ve written up some code taking advantage of the new copy blob and get committed block list methods in Windows Azure storage.  This code comes in the form of a modified copy of the sample storage client library that ships in the Windows Azure SDK.  For the impatient, you can go ahead and download the code.

To use the new features, you’ll need this code or your own code like it, because the SDK sample hasn’t yet been updated to include the new functionality.  You’ll also need to run against a storage account in the cloud, because the development storage in the SDK has also not yet been updated to match the latest bits in the cloud.

Disclaimer

The standard level of quality of sample code I write is not terribly high, and this is no exception.  Please take this code for what it is, a sample of how to use the new functionality.  An updated storage client library in the future is the right place to get higher quality code.

The New Features

Copy blob

This method has been added.  It allows you to copy from one blob to another within the same container of the same account.  Takes the form over the wire as an HTTP PUT with no body but a header specifying the source blob name.

For full details, see the MSDN documentation for Copy Blob.

Get block list

This method was modified since earlier releases to include support for a blocklisttype query parameter.  Setting different values for this parameter lets you get the list of committed blocks, uncommitted blocks, or all of them combined.

For full details, see the MSDN documentation for Get Block List.

The New Methods

To take advantage of these new features, I added the following five methods to the sample storage client library:

public abstract bool CopyBlob(string to, string from, bool overwrite);
public abstract List<string> GetBlockList(string blobName, string eTag);
public abstract List<string> GetCommittedBlockList(string blobName, string eTag);
public abstract List<string> GetUncommittedBlockList(string blobName, string eTag);
public abstract List<string> GetAllBlockList(string blobName, string eTag);

In the code, these are declared in BlobStorage.cs.  They’re implemented in RestBlobStorage.cs with the help of a few additional constants in RestHelpers.cs.

There’s also a console application included that tests the new functionality.  To use it, modify Program.cs to use a valid storage account and key.  The output of the test program should look like this:

Creating container.
Creating big blob (5MB).
Done.
Committed blocks:
        AAAAAA==
        AQAAAA==
        AgAAAA==
        AwAAAA==
        BAAAAA==
Copying blob...
Done.
Committed blocks:
        AAAAAA==
        AQAAAA==
        AgAAAA==
        AwAAAA==
        BAAAAA==
Deleting test container.

[UPDATE 6/5/2009] The test program now generates some more output based on entity group transactions.  See the next blog post ("Sample Code for Batch Transactions in Windows Azure Tables”) for details.

Miscellaneous Details

As part of the release of the new storage code, a versioning header has been introduced.  Setting the header x-ms-version to 2009-04-14 unlocks the new functionality, so I added a line to Utilities.CreateHttpRequest in RestHelpers.cs that adds this header to all HTTP requests.

I also exposed for the first time the Get Block List functionality, though it existed in the storage client library in a private method.  I changed the way this works… it used to return a list of integers, each representing the size of a block.  Now it returns a list of strings, each of which is the name of a block.  This seems more natural to me, but be aware of that change.

I did nothing smart in this code to take advantage of the new functionality to get the committed block list for a blob.  The most obvious use of this would be to make a smart blob uploader that automatically resumes an upload without having to keep local state to know which blocks were successfully uploaded.  (It can just query the server.)

Download

You can find the full source code (updated storage client library and console test application) here: http://smarxblogstorage.blob.core.windows.net/files/StorageMayFeatures.zip.  Enjoy!


Comments

Floating on Clouds
2009-06-01 02:56 GMT
This rocks Steve. Thank you!
2009-06-03 05:05 GMT
Great work. The code is very useful. Thanks for sharing
Michael
2009-06-18 06:49 GMT
Thank you for sharing!
2009-09-10 05:13 GMT
The code is very useful. Thanks for sharing
2009-09-10 05:14 GMT
This rocks Steve. Thank you!
2009-09-10 12:56 GMT
I did nothing smart in this code to take advantage of the new functionality to get the committed block list for a blob.
2009-11-03 18:10 GMT
thank you for the great help
the wedding rings are very nice
2009-11-03 18:10 GMT
thank you for the great help
the wedding rings are very nice
Thanks
2009-11-03 18:10 GMT
thank you for the great help
the wedding rings are very nice
Thanks
2009-11-03 18:11 GMT
thank you for the great help
the wedding rings are very nice
Thanks
2009-11-05 03:39 GMT
this code works very well for me. This is just amazing how this works. Thanks for the great info.
2009-11-08 10:56 GMT
thank you for the great help
the wedding rings are very nice
2009-11-09 17:20 GMT
Now this is really helpful. Thanks a bunch!
2009-11-11 13:19 GMT
really good work
2009-11-11 13:20 GMT
nice job
2009-11-11 13:20 GMT
very interesting
2009-11-12 19:22 GMT
Another very helpful tip, thanks Steve
2009-11-12 19:23 GMT
Thanks for the example
2009-11-13 10:30 GMT
very nice
2009-11-15 09:17 GMT
Copy Blob,thanks for sharing.Wonderful codes with beautiful wedding rings.Successfully uploaded,easy and simple to understand.Thanks.
2009-11-16 17:42 GMT
Simply awesome. This is where you can start showing the real power of Windows Azure. As an Australian with 250ms latency to the US I'm really looking forward to seeing how this performs.
2009-11-18 09:45 GMT
nice features and good information about this
2009-11-19 09:23 GMT
thanks
2009-11-19 09:24 GMT
thanks man
2009-11-20 09:55 GMT
THX
2009-11-21 22:11 GMT
Downloaded the Source code, great stuff!
2009-11-21 22:11 GMT
Downloaded the Source code, great stuff!
2009-11-21 22:11 GMT
Downloaded the Source code, great stuff!
2009-11-21 22:12 GMT
Downloaded the Source code, great stuff!
2009-11-21 22:12 GMT
Downloaded the Source code, great stuff!
2009-11-21 22:12 GMT
Downloaded the Source code, great stuff!

Thank you
2009-12-01 09:07 GMT
thanks
2009-12-02 06:41 GMT
thanks for the code, helped me a lot
2009-12-02 20:44 GMT
thx for info und lot of info
2009-12-02 20:45 GMT
thx for info und lot of info
2009-12-03 09:04 GMT
thanks good article
Anonymous
2009-12-03 11:02 GMT
Aloha!vuvp! http://chsairyw.com aghtv hpcvp http://klssmjim.com uyeit aodtm http://yhayqagr.com vqhqj gsydk http://aikowcla.com jeoxl wrtbc http://oedkcghh.com mqjrn lburv
Anonymous
2009-12-03 11:03 GMT
[URL=http://dejynxzh.com]kdhgo qgtjm[/URL] [URL=http://batsbhiy.com]viwqr xjbho[/URL] [URL=http://vzkstzvk.com]fqddu xnxfl[/URL] [URL=http://cdjtnozv.com]jgxjk hflga[/URL] [URL=http://thqwtylv.com]aovzr zdsll[/URL]
Anonymous
2009-12-03 11:03 GMT
[URL=http://hlgqtajt.com]jnyhr garcz[/URL]
2009-12-03 12:16 GMT
Vielen Dank für diese Infos
2009-12-05 01:27 GMT
Downloaded the Source code, great stuff
Tom
2009-12-08 12:32 GMT
Thanks for the example
2009-12-08 14:23 GMT
thanks for the code it works well
2009-12-08 21:59 GMT
Thx for sharing!
2009-12-08 22:00 GMT
Thx for sharing
2009-12-11 08:39 GMT
Thanks good post nice blog
2009-12-15 00:58 GMT
Thank you for the sensible critique. Me & my neighbour were preparing to do some research about that. We got a good book on that matter from our local library and most books where not as influensive as your information. I am very glad to see such information which I was searching for a long time.This made very glad Smile
2009-12-15 20:39 GMT
It´s a very critical article but necessary to my opinion
2009-12-16 15:19 GMT
Nice
2009-12-17 07:56 GMT
this code works awesome for me. This is just amazing how this works. Thanks for the great info. cheers!!!
2009-12-17 07:56 GMT
this code works awesome for me. This is just amazing how this works. Thanks for the great info. cheers!!!
2009-12-17 07:56 GMT
this code works awesome for me. This is just amazing how this works. Thanks for the great info. cheers!!!
2009-12-17 10:47 GMT
very ncies article..
2009-12-18 20:11 GMT
Very nice Blog.Best regards mj
2009-12-21 06:19 GMT
Simply awesome. This is where you can start showing the real power of Windows Azure. As an Australian with 250ms latency to the US I'm really looking forward to seeing how this performs.
2009-12-21 13:57 GMT
Hello Steve, Thanks a lot this is really informative and useful.
2009-12-21 18:57 GMT
As part of the release of the new storage code, a versioning header has been introduced.  . .
2009-12-22 09:42 GMT
I did nothing smart in this code to take advantage of the new functionality to get the committed block list for a blob. - awesome job!!!

2009-12-25 16:34 GMT
thanks so muchh
2009-12-27 08:27 GMT
Windows Azure Blob enables applications to store and manipulate large objects and files in the cloud.
2009-12-29 08:02 GMT
this code works awesome for me. This is just amazing how this works. Thanks for the great info. cheers!!!

2009-12-31 19:02 GMT
Thank you for sharing
2010-01-02 17:08 GMT
I did nothing smart in this code to take advantage of the new functionality to get the committed block list for a blob.
2010-01-02 18:53 GMT
I did nothing smart in this code to take advantage of the new functionality to get the committed block list for a blob.
2010-01-03 18:10 GMT
thank you for the great help
the wedding rings are very nice
Thanks
2010-01-03 22:57 GMT
Azure works great! It provides a LOT of functionality and provides core functionality such as virtually unlimited online file storage and simple database storage. I think developers and companies will appreciate the rich set of tools and services it offers as well...
2010-01-05 07:37 GMT
thank you for the great help
the wedding rings are very nice
Thanks
2010-01-06 06:21 GMT
I was very pleased to find this site.I wanted to thank you for this great read!! I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post.
regards,
2010-01-06 06:21 GMT
I was very pleased to find this site.I wanted to thank you for this great read!! I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post.
2010-01-07 06:29 GMT
I was very pleased to find this site.I wanted to thank you for this great read!! I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post.
2010-01-07 16:56 GMT
thanks my friend
2010-01-07 17:36 GMT
I need the ASCII code for change the numeric digits of my computer system . Can you deal in this boundary ?
2010-01-07 17:36 GMT
I need the ASCII code for change the numeric digits of my computer system . Can you deal in this boundary ?
2010-01-07 17:36 GMT
I need the ASCII code for change the numeric digits of my computer system . Can you deal in this boundary ?
2010-01-07 17:36 GMT
I need the ASCII code for change the numeric digits of my computer system . Can you deal in this boundary ?
2010-01-07 17:37 GMT
I need the ASCII code for change the numeric digits of my computer system . Can you deal in this boundary ?
2010-01-08 03:46 GMT
thank you
2010-01-08 07:11 GMT



thanks for article

all a successful 2010!!

Marco
2010-01-08 07:11 GMT



thanks for article

all a successful 2010!!

Marco
2010-01-08 15:06 GMT
thank you for the great help
the wedding rings are very nice
2010-01-09 22:42 GMT
comment3,
2010-01-10 08:32 GMT
Thank you very much!
Your article is very useful!
2010-01-10 10:02 GMT
If I have documents sitting in a cloud storage such as Google Docs, why do I need to back it up to a different cloud storage such as Azure Storage?
2010-01-11 18:10 GMT
Copy Blob,thanks for sharing.Wonderful codes with beautiful wedding rings.Successfully uploaded,easy and simple to understand.Thanks.
2010-01-12 17:28 GMT
I was very pleased to find this site.I wanted to thank you for this great read!! I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post.
2010-01-13 07:54 GMT
It is now possible to perform a transaction across entities stored within the same table and same partition (i.e., entities having the same partition key value). An application can now atomically perform multiple Create/Update/Delete operations across multiple entities in a single batch request to the storage system, as long as the entities in the batch operation have the same partition key value and are in the same table.
2010-01-13 16:45 GMT
The Windows Azure Blob features that allow customers to manipulate files as large as 50 GB have been updated.
2010-01-14 11:42 GMT
satotravel.net – a official website that presents economical rates for the Military Travel, Sato Travel agency and Sato Travel Europe.
2010-01-14 21:26 GMT
That simple code is rox!!!
2010-01-14 21:26 GMT
That simple code is rox!!!
2010-01-14 21:27 GMT
That simple code is rox!!!
2010-01-14 21:27 GMT
That simple code is rox!!!
Amazing.
Anonymous
2010-01-15 06:47 GMT
Thanks for reporting this. We have had some similar reports, but never quite the same case.
2010-01-16 15:29 GMT
thanks _:)
2010-01-17 16:19 GMT
Nice place.
Code is fantastic.
2010-01-18 14:56 GMT
thank you...
2010-01-19 23:15 GMT
Thanks for sharing such a wonderful information with us.
2010-01-21 06:20 GMT
Microsoft will revise the wording on its Azure cloud service level agreements following a trip into a Kafkaesque world of secrecy.
2010-01-21 06:20 GMT
Microsoft will revise the wording on its Azure cloud service level agreements following a trip into a Kafkaesque world of secrecy.
2010-01-21 06:20 GMT
Microsoft will revise the wording on its Azure cloud service level agreements (SLAs) following a trip into a Kafkaesque world of secrecy.
2010-01-21 06:20 GMT
Microsoft will revise the wording on its Azure cloud service level agreements (SLAs) following a trip into a Kafkaesque world of secrecy.
2010-01-21 06:21 GMT
l when we are planning to accomplish a lit bit tough and complicated task in a very short time and in efficient way.
2010-01-21 08:01 GMT
thanks for artice.
2010-01-21 14:05 GMT
In Q3 2009 the company said that Azure will be more than just a PaaS cloud, hosting (Hyper-V) virtual machines pretty much like Amazon does with its Elastic Computing Cloud (EC2). Which makes it a hybrid IaaS + PaaS cloud. But Microsoft didn’t clarify if the IaaS offering would be launched at the beginning of 2010 with the PaaS one.
2010-01-22 17:06 GMT
I was very pleased to find this site.I wanted to thank you for this great read!! I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post.
2010-01-23 14:25 GMT
At Tradeslot, we design large-scale business-to-business and government-to-business auction platforms. Unlike other auction platforms, we have built a combinatorial auction platform that allows bidders to place expressive bids. Adslot uses the same intellectual property as the auction platform and enables media publishers to sell ad inventory. e
2010-01-24 08:57 GMT
If I have documents sitting in a cloud storage such as Google Docs, why do I need to back it up to a different cloud storage such as Azure Storage?
2010-01-24 19:04 GMT
This is great, but a little to complicated for me I guess. But thxt for the tip anyway.
2010-01-25 09:08 GMT
Users of Microsoft’s first ever Cloud operating system will undoubtedly welcome an application designed to enhance the platform, especially to simplify the management of blobs.
2010-01-26 16:26 GMT
It´s a very critical articleGreat work
2010-01-26 16:26 GMT
It´s a very critical article.Great work.
2010-01-27 12:35 GMT
thanks my friend
2010-01-27 21:27 GMT
i also thx for sharing this
2010-01-27 21:27 GMT
i also thx for sharing this
2010-01-28 07:24 GMT
This rocks Steve. Thank you!
2010-01-29 16:17 GMT
Thanks good work
2010-01-30 08:44 GMT
Well, this is a very valuable post. Thanks for the information you provided. It would be great if got more post like this. I appreciate it.
2010-01-30 10:17 GMT
nice place like photo lke this.
2010-01-30 10:17 GMT
nice place like photo lke this.
2010-01-30 10:17 GMT
nice place like photo lke this.
2010-01-31 21:56 GMT
Kombi Servisleri

http://www.kombi-servisleri.com
2010-02-01 05:00 GMT
Ok Steve Marx ,I will download and check it now
2010-02-01 05:00 GMT
Ok Steve Marx ,I will download and check it now
2010-02-01 11:36 GMT
Thanks good work
2010-02-02 04:13 GMT
Thanks for sharing such good piece of info...now going to download it!!
2010-02-02 04:13 GMT
Thanks for sharing such good piece of info...now going to download it!!
2010-02-02 04:14 GMT
Thanks for sharing such good piece of info...now going to download it!!
2010-02-02 05:17 GMT
Hey great information man.....thanks for sharing this with us!!!
2010-02-02 05:17 GMT
Hey great information man.....Thanks for sharing this with us!!!
2010-02-02 05:18 GMT
Hey great information man.....Thanks for sharing this with us!!!
2010-02-02 11:21 GMT
Hey Dude..thats great info..... really you have done an amazing job by sharing this informative data.

thanks
stela...
2010-02-02 11:22 GMT
Hey Dude..thats great info..... really you have done an amazing job by sharing this informative data.

thanks
stela...
2010-02-02 11:22 GMT
Hey Dude..thats great info..... really you have done an amazing job by sharing this informative data.

thanks
stela...
2010-02-02 11:22 GMT
Hey Dude..thats great info..... really you have done an amazing job by sharing this informative data.

thanks
stela...
2010-02-02 13:10 GMT
Very informative article, thanks for the heads up!
2010-02-02 13:11 GMT
Very informative article, thanks for the heads up!
2010-02-02 18:36 GMT
l when we are planning to accomplish a lit bit tough and complicated task in a very short time and in efficient way.
2010-02-02 21:35 GMT
keep up the good work!
2010-02-02 22:11 GMT
i agree with you. nice article and some helpful infos
2010-02-02 22:11 GMT
i agree with you. nice article and some helpful infos
2010-02-03 04:45 GMT
Here first the writer describes the topic very keenly that continuation tokens are the way you can pick up a query where it left off. In the paging example, you’re explicitly querying for a subset of the results.When you’re ready for the next page, you query again, passing in the continuation token you received from the first query and it leads t the next page very quickly. As a result of this the next set of results and the next continuation token.and this procedure goes on again and again.
2010-02-03 13:48 GMT
Yes, this article is very helpful and infoative for me. Thanks
2010-02-03 15:19 GMT
thanks good post
2010-02-04 01:37 GMT
I strongly agree and have forwarded it to a number of associates who found it very useful, keep up the good work.
2010-02-04 01:37 GMT
I strongly agree and have forwarded it to a number of associates who found it very useful, keep up the good work.
2010-02-05 00:01 GMT
i agree with you. nice article and some helpful infos
2010-02-05 00:01 GMT
i agree with you. nice article and some helpful infos
2010-02-05 13:05 GMT
Looks like a tricky tool :-)
2010-02-05 13:05 GMT
Looks like a tricky tool.
2010-02-09 07:32 GMT
I would like to thank you for the efforts you have made in writing this article and i am hoping the same best work from you in the future as well.

Add a comment

Your name:
Your URL
(will be linked from your name):
Your comment: