Fri, 14 May 2010

Serving Your Website From a Windows Azure Drive

[UPDATE 12/14/2010] Now that SDK 1.3 and above support full IIS, I don’t see why my HWC Worker solution is needed anymore. I’m leaving this and other blog posts up for those interested, but I’m not fixing any bugs. (There’s a typo in one bit of documentation and a bug preventing VHD uploads larger than 2GB. There are probably other issues I don’t know about, so consider yourself warned.)

For this week’s episode of Cloud Cover, Ryan and I showed how to mount a Windows Azure drive and serve your website from it using my Hosted Web Core Worker Role. I’ve released a new Visual Studio 2010 solution called HWCWorker_Drive_source.zip that mounts a snapshot of a Windows Azure Drive and points Hosted Web Core at it. You can download it over on the Hosted Web Core Worker Role project on Code Gallery.

To use the project, you need to store your web content in a VHD, upload it to the cloud, take a snapshot of the VHD, and then configure the project to use that snapshot. To make changes, just upload the new version of the VHD, take a new snapshot, and change the configuration setting in the project.

In the rest of this post, I’ll walk through how the various steps work.

Creating a VHD on Windows 7 or Windows Server

Yeah, you could be a wuss and use the GUI, but for those of you who love the command-line (as I do), I’ve included three scripts along with the code that create, mount, and unmount VHDs.

The first script, createvhd.cmd, creates a VHD of the specified size and assigns it a drive letter:

@echo off
if "%2"=="" goto usage
set driveletter=%3
if driveletter=="" set driveletter=v
echo create vdisk file="%~f1" maximum=%2 type=fixed > "%TEMP%\script.txt"
echo select vdisk file="%~f1" >> "%TEMP%\script.txt"
echo attach vdisk >> "%TEMP%\script.txt"
echo create partition primary >> "%TEMP%\script.txt"
echo assign letter=%driveletter% >> "%TEMP%\script.txt"
echo format fs=ntfs label=vhd quick >> "%TEMP%\script.txt"
diskpart -s %TEMP%\script.txt
echo VHD should be available shortly at "%driveletter%:"
goto exit
:usage
echo USAGE: createvhd.cmd <path-to-vhd> <size-in-megabytes>
:exit

This also mounts the drive, so you can go copy in your web content. To unmount the drive (which you must do before uploading it to the cloud), you can use unmountvhd.cmd:

@echo off
if "%1"=="" goto usage
echo select vdisk file="%~f1" > "%TEMP%\script.txt"
echo detach vdisk >> "%TEMP%\script.txt"
diskpart -s %TEMP%\script.txt
goto exit
:usage
echo USAGE: unmountvhd.cmd <path-to-vhd>
:exit

And finally, to mount it again (to make changes), you can use mountvhd.cmd:

@echo off
if "%1"=="" goto usage
echo select vdisk file="%~f1" > "%TEMP%\script.txt"
echo attach vdisk >> "%TEMP%\script.txt"
diskpart -s %TEMP%\script.txt
echo VHD should be available shortly at the drive letter you assigned when you created it.
goto exit
:usage
echo USAGE: mountvhd.cmd <path-to-vhd>
:exit

Uploading and Snapshotting the VHD

One of the projects included in the HWCWorker_Drive_source.zip solution is a command-line tool called UploadAndSnapshot.exe. It does exactly what the name suggests: it uploads a VHD (or really any file) as a page blob in Windows Azure storage, takes a snapshot of that blob, and then returns the full URL to the snapshot. You can take the output of this command and put it directly into the DriveSnapshotUrl configuration setting of the cloud project. This is the snapshot that the application will mount and serve your website from.

UploadAndSnapshot uses the technique described on the Windows Azure storage team blog to efficiently upload the VHD by including only the non-zero pages. This is important both for efficiency reasons (less bytes to upload) and for cost reasons (only charged for the pages uploaded).

The usage of UploadAndSnapshot is simple. Run it without any parameters to see the usage:

Usage UploadAndSnapshot.exe <connectionstring> <path-to-local-file> <url-to-uploaded-blob>
e.g. UploadAndSnapshot.exe DefaultEndpointsProtocol=http;AccountName=myaccount;AccountKey=1F84D...
    c:\mydrive.vhd drives/mydrive.vhd

Mounting and Using the VHD

To use a Windows Azure drive, you just need to create a local cache and then mount the drive. The following code handles this in HWCWorker_Drive in OnStart():

var localCache = RoleEnvironment.GetLocalResource("DriveCache");
CloudDrive.InitializeCache(localCache.RootPath.TrimEnd('\\'), localCache.MaximumSizeInMegabytes);

var cloudAccount = CloudStorageAccount.Parse(
    RoleEnvironment.GetConfigurationSettingValue("DriveConnectionString"));
drive = cloudAccount.CreateCloudDrive(RoleEnvironment.GetConfigurationSettingValue("DriveSnapshotUrl"));

var drivePath = drive.Mount(localCache.MaximumSizeInMegabytes, DriveMountOptions.None);

Then in OnStop() I call drive.Unmount().

Using the Development Fabric

Windows Azure drives work completely differently in the development fabric. Instead of mounting an actual VHD, development storage simulates it by creating a directory on the local filesystem, and it can not be used to mount a VHD located in the cloud. I haven’t played around with doing this on development storage enough to talk through it yet, so I’ll probably follow up with a post in the future about how to do this.

For now, my solution only works in the cloud.

Known Issues

After deploying this solution to the cloud, changing the snapshot configuration setting seems to work inconsistently. Sometimes it works fine, but sometimes a role instance will cycle, continually restarting. You can, of course, deploy the application again with the new snapshot to staging, and then swap it into production.

~~I need to add some logging to debug this further. If I figure out the mystery, I’ll update this post, but I wanted to get the code out now.~~

[UPDATE 5/14/2010] After a number of attempts to reproduce the error, I think it may have been caused by a simple error on my part specifying a bogus snapshot. If someone sees this problem, please let me know so I can investigate.

More Information

If you’re interested in learning more about Windows Azure drives, I recommend reading the following blog posts on the Windows Azure storage team blog:

You should also read the Windows Azure Drives whitepaper.

Get the Code

You can download the HWCWorker_drive_source.zip package over on the Hosted Web Core Worker Role project on Code Gallery.