blog.smarx.com

Steve Marx's blog about cloud development

Sample Code for Batch Transactions in Windows Azure Tables

As a follow-up to my earlier post about the new blob storage features, in this post, I’ll show you how to take advantage of the entity group transactions functionality that was recently rolled out to Windows Azure tables.

I’ve updated the project (modified storage client library and test program) I announced in the last post, so go ahead and download the code again.  To use the new features, you’ll need this code or your own code like it, because the SDK sample hasn’t yet been updated to include the new functionality.  You’ll also need to run against a storage account in the cloud, because the development storage in the SDK has also not yet been updated to match the latest bits in the cloud.

What are “Entity Group Transactions”?

Entity group transactions is a feature that allows you to make changes to multiple entities as part of the same operation, as long as all the entities are part of the same “entity group” (for now, that just means the same partition).  Here’s a more complete description from the “Programming Table Storage” whitepaper:

For the entities stored within the same table and same partition (i.e., they have the same partition key value), the application can atomically perform a transaction involving those entities. This allows the application to atomically perform multiple Create/Update/Delete operations across multiple entities in a single batch request to the storage system, as long as all the entities have the same partition key value and are in the same table. Either all the entity operations succeed in the single transaction or they all fail, and snapshot isolation is provided for the execution of the transaction. In addition, all other queries executing in parallel at the same time will not see the result of the transaction, since they will be working off a prior snapshot. Queries will only see the result of the transaction, once it has fully successfully committed.

Entity Group Transaction will require the use of the version header with the version set to "2009-04-14" or later.

I recommend the MSDN documentation for entity group transactions to see exactly what it looks like in the REST API.

How Do I Use Them?

Using this functionality is simple, because it’s already built into the ADO.NET Data Services protocol that we use in Windows Azure Tables.  I took two simple steps to take advantage of the new functionality:

  1. I modified the storage client library to put the x-ms-version header (with value 2009-04-14) on all web requests to table storage.
  2. I wrote some code using the SaveChangesOptions.Batch flag on my SaveChanges call.

That’s it!  The updated test program in the download should show the following output:

Creating container.
Creating big blob (5MB).
Done.
Committed blocks:
        AAAAAA==
        AQAAAA==
        AgAAAA==
        AwAAAA==
        BAAAAA==
Copying blob...
Done.
Committed blocks:
        AAAAAA==
        AQAAAA==
        AgAAAA==
        AwAAAA==
        BAAAAA==
Deleting test container.
Adding three entities (same table, same partition)...
Done.
Updating one entity and deleting another (same table, same partition)...
Done.
Deleting two entities (same table, same partition)...
Done.
Adding two entities (different table, different partition)...
        Got error: CommandsInBatchActOnDifferentPartitions: 1:All commands in a
batch must operate on same entity group.
Done.
Adding two entities (same table, different partition)...
        Got error: CommandsInBatchActOnDifferentPartitions: 1:All commands in a
batch must operate on same entity group.
Done.

Download

Download the code to see exactly what I did and to see both positive and negative tests showing that all the actions as part of the batch transaction need to operate on entities in the same partition of the same table.


Comments

ChrisM
2009-08-29 20:09 GMT
Is there any benefit to creating a new Context(account) for every operation, or is it OK to re-use an existing context for different operations?
Rahul Bandopadhyaya
2009-11-20 12:54 GMT
thanks steve for this valuable post...

i have one question, if i have for example in table storage for one table data in GBs or TBs, how the storageclient handles the fetching of the entities. i know that using the ExecuteAll method exposed by TableStorageDataServiceQuery we can fetch entities more then 1000 bu thow it is handled once fetched. certainly we cant keep the entire list of entities in memory (when the data is in GBs and TBs). is there any option of fetching data in batches in storageclient?
Rahul Bandopadhyaya
2009-11-20 12:54 GMT
thanks steve for this valuable post...

i have one question, if i have for example in table storage for one table data in GBs or TBs, how the storageclient handles the fetching of the entities. i know that using the ExecuteAll method exposed by TableStorageDataServiceQuery we can fetch entities more then 1000 bu thow it is handled once fetched. certainly we cant keep the entire list of entities in memory (when the data is in GBs and TBs). is there any option of fetching data in batches in storageclient?
Rahul Bandopadhyaya
2009-11-20 12:54 GMT
thanks steve for this valuable post...

Rahul Bandopadhyaya
2009-11-20 12:56 GMT
thanks steve for this valuable post...

Anonymous
2009-12-03 11:02 GMT
Aloha!ktiu! http://utugzphm.com psjxz mnxib http://qwozosgy.com xzfxy kskwi http://okrcasor.com bhgyy bcssv http://vvgzwvel.com bmtgq gcmbg http://tfccvkoa.com sqbaz blwsf
Anonymous
2009-12-03 11:03 GMT
[URL=http://rkiwwvuh.com]qpjad wvffn[/URL] [URL=http://kcmxmpka.com]ntlmg fewwo[/URL] [URL=http://fhcjdeva.com]rgvif xfyuf[/URL] [URL=http://jonjgwuf.com]lwbks unqnl[/URL] [URL=http://qyyqvjym.com]ghvaw vppxj[/URL]
Anonymous
2009-12-03 11:03 GMT
[URL=http://vpjsxudn.com]zvcxu dykbs[/URL]
2009-12-15 05:16 GMT
Thank you for the sensible critique. Me & my neighbour were preparing to do some research about that. We got a good book on that matter from our local library and most books where not as influensive as your information. I am very glad to see such information which I was searching for a long time.This made very glad Smile
2009-12-30 17:06 GMT
This topic lists some examples of using the PHPAzure SDK. Other features are available in the download package, as well as a detailed API documentation of those features.
2010-01-21 07:26 GMT
One option that I saw recently is Maarten Balliauw’s PHP library. Currently it only supports blob storage, but he tells me he’s working on tables and queues next.
2010-01-21 07:26 GMT
I figured now was a good time to introduce the concept of paging to my blog.
2010-01-21 07:27 GMT
Now that we’ve added support for PHP, many people are wondering where the storage client library is for PHP. 
2010-01-21 07:27 GMT
Windows Azure storage is accessible via a REST API, but in practice very few people write code directly against the REST endpoint, instead using the sample storage client library that ships with the Windows Azure SDK.
2010-01-21 07:27 GMT
This diagram shows PHP serving all the UI, but in actuality, both ASP.NET and PHP are rendering UI, as you can see at http://tweval.com/mix09-smarx and http://tweval.com/index.php?tag=mix09-smarx (note the bottom right corner of the page).
2010-01-23 14:45 GMT
I figured now was a good time to introduce the concept of paging to my blog. rewr
2010-02-02 09:39 GMT
In the world of business promotion and SEO techniques, Blog commenting services have created a unique space for themselves. The ‘Big Kingdom’ of these services is open to one and all and can be accessed at very nominal prices online seo services.
2010-02-04 09:55 GMT
Its soo good to see this information in your post, I was looking the same but there was not any proper resource, thanx now I have the thing which i was looking for my work.
2010-02-04 09:55 GMT
Its soo good to see this information in your post, I was looking the same but there was not any proper resource, thanx now I have the thing which i was looking for my work.
2010-02-09 06:08 GMT
These modifications are really helpful. I have for example in table storage for one table data in GBs or TBs, how the storageclient handles the fetching of the entities. i know that using the ExecuteAll method exposed by TableStorageDataServiceQuery we can fetch entities more then 1000 bu thow it is handled once fetched.

Add a comment

Your name:
Your URL
(will be linked from your name):
Your comment: