Question

I have a WCF service hosted on Windows Azure as a “cloud service.” When the service starts, it needs to populate data from files/disk to its memory so it is accessed fast (cached in other words). Right now I’m using like C:\Documents\Filestoprocess folder so that the WCF calls the folder and populates data data in that folder in its memory. I have like 5,000 small files. How do I do this in Azure? Is there a folder path that I can call within the WCF so that the WCF calls these files and opens each files and saves each data in the files? I’m not really looking for complicated Blob access through network using bandwidth. I’m looking for simple disk I/O access to these files from the WCF “cloud service” that is running on its own public web address.

Answer 1

Blob access is not complex. In fact, you could do a single download of a zip file from blob storage to local disk, unzip it, then prime your wcf service from those 5,000 small files.

Check out this msdn page documenting DownloadBlobToFile(). The essential parts:

CloudBlobClient blobClient = 
        new CloudBlobClient(blobEndpoint, new StorageCredentialsAccountAndKey(accountName, accountKey));

    // Return a reference to the blob.
    CloudBlob blob = blobClient.GetBlobReference("mycontainer/myblob.txt");

    // Download the blob to a local file.
    blob.DownloadToFile("c:\\mylocalblob.txt");

Now: I don’t agree with saving to the root folder on C:. Rather, you should grab some local storage (easily configurable). Once you configure local storage in your role configuration, just ask the role environment for it, and ask for root path:

var localResource = RoleEnvironment.GetLocalResource("mylocalstorage");
var rootPath = localResource.RootPath;

Note: As @KingPancake mentioned, you could use an Azure drive. However: remember that an Azure drive can only be writeable by one instance. You’d need to make additional snapshots for your other instances. I think it’s much simpler for you to go with a simple blob, copy your files down (either as single zip or individual files), and go from there.

You mentioned concern with network+bandwidth. You don’t pay for bandwidth within the same data center. Also: It’s extremely fast: 100Mbps per core. So even with a Small instance, you’ll have your files copied down very quickly, moreso when you go to larger instance sizes.

One last thought: The only other ways to gain access to your 5,000 files, without using blob storage or Azure Drives (which are mounted as vhd’s in blob storage) would be to either download the files from an external source or bundle them with your Windows Azure package (and then they’d show up in your app’s folder, under whatever subfolder you stuck them in). Bundling has two downsides:

• Longer time to upload your deployment package due to added size
• Inability to change any of the individual files without redeploying the package.

By storing in a blob, you can easily change one (or all) of your small files without redeploying your code – you’d just need to signal it to either re-read from blob storage or restart the instances so they automatically download the new files.