Question

I seem to be getting a 403:

HTTP/1.1 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

This happens when I use set the HttpWebRequest.DefaultCachingPolicy in the app.config as follows:

<system.net>
 <requestCaching defaultPolicyLevel="Default" isPrivateCache="false">
  <defaultHttpCachePolicy policyLevel="Default"/>
 </requestCaching>
</system.net>

I’m doing this because I have legacy code which I do not control that is calling an API to blob storage that I have provided as if it were a file system (up to 58 identical calls per file). Clearly this is not ideal. Using default HTTP style caching is a behavior I would like as it would cause my application to only download the file when it’s modified.

The issue seems to occur every other request (e.g. it appears to be happening when the request is cached and the server is checking to see if the server content has changed).

The only difference between the requests that fail and the requests that succeed seems to be the inclusion of:

If-None-Match: "<a blob etag>"
If-Modified-Since: <a date>

I’ve looked at the code for the .net API (which I’m using) on github for 1.7.1 and assuming it hasn’t changed from SDK 1.6 (what I’m currently using), it should work fine.

Any help is very appreciated

UPDATE:
I’ve written some repro code to assist:

Uses: .NET 4.0, Windows Azure SDK 1.6

using System;
using System.Net;
using System.Net.Cache;
using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.StorageClient;

namespace AzureStorageProb
{
    class Program
    {
        static void Main(string[] args)
        {
            const string accountKey = "<azure storage account key>";
            const string account = "<azure storage account name>";
            const string testBlob = "<blob path to test file>";
            var cloudStorageAccount = 
               new CloudStorageAccount(
                  new StorageCredentialsAccountAndKey(account, accountKey),
                  useHttps: true);
            var cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();
            HttpWebRequest.DefaultCachePolicy = 
               new HttpRequestCachePolicy(HttpRequestCacheLevel.Default);

            try
            {
                var blob = cloudBlobClient.GetBlobReference(testBlob);
                blob.FetchAttributes();
                blob.DownloadByteArray();
                Console.WriteLine("First attempt worked!");
            }
            catch (StorageClientException ex)
            {
                Console.WriteLine(ex);
            }

            try
            {
                var blob = cloudBlobClient.GetBlobReference(testBlob);
                blob.FetchAttributes();
                blob.DownloadByteArray();
                Console.WriteLine("Second attempt worked!");
            }
            catch (StorageClientException ex)
            {
                Console.WriteLine(ex);
            }
            Console.ReadKey();
        }
    }
}

Answer 1

So it turns out the culprit is BlobRequest.SignRequest(request, creds); A quick peek at the documentation indicates that the If-None-Match and If-Modified-Since are both used as part of the calculation for the Authentication header that call adds. Because those headers are added AFTER the Authentication header is computed by WINInet (a presumption). Either way the end result is that the checksum provided by the Authentication header is now invalid. Thus causing the 403 Forbidden.

Work Arounds:

• Manually cache items locally
• Fix the Calling code (probably the best long term solution)
• Use the rest interface VERY carefully (experimentation has shown this to be dangerous for the same reasons, using the ‘Shared Key Lite’ gets around this in some circumstances)
• Use a regular WebRequest to get files (this means you have to expose them to the internet, either via shared key or otherwise)

I’m not going to say that Azure Blob Storage is broken… but I will say that it is not truly RESTful as it cannot comply with HTTP semantics correctly.