Question

For the windows azure queues the scalability target per storage is supposed to be around 500 messages / second (http://msdn.microsoft.com/en-us/library/windowsazure/hh697709.aspx). I have the following simple program that just writes a few messages to a queue. The program takes 10 seconds to complete (4 messages / second). I am running the program from inside a virtual machine (on west-europe) and my storage account also is located in west-europe. I don’t have setup geo replication for my storage. My connection string is setup to use the http protocol.

       // http://blogs.msdn.com/b/windowsazurestorage/archive/2010/06/25/nagle-s-algorithm-is-not-friendly-towards-small-requests.aspx
        ServicePointManager.UseNagleAlgorithm = false;

        CloudStorageAccount storageAccount=CloudStorageAccount.Parse(ConfigurationManager.AppSettings["DataConnectionString"]);

        var cloudQueueClient = storageAccount.CreateCloudQueueClient();

        var queue = cloudQueueClient.GetQueueReference(Guid.NewGuid().ToString());

        queue.CreateIfNotExist();
        var w = new Stopwatch();
        w.Start();
        for (int i = 0; i < 50;i++ )
        {
            Console.WriteLine("nr {0}",i);
            queue.AddMessage(new CloudQueueMessage("hello "+i));    
        }

        w.Stop();
        Console.WriteLine("elapsed: {0}", w.ElapsedMilliseconds);
        queue.Delete();

Any idea how I can get better performance?

EDIT:

Based on Sandrino Di Mattia’s answer I re-analyzed the code I’ve originally posted and found out that it was not complete enough to reproduce the error. In fact I had created a queue just before the call to ServicePointManager.UseNagleAlgorithm = false; The code to reproduce my problem looks more like this:

        CloudStorageAccount storageAccount=CloudStorageAccount.Parse(ConfigurationManager.AppSettings["DataConnectionString"]);

        var cloudQueueClient = storageAccount.CreateCloudQueueClient();

        var queue = cloudQueueClient.GetQueueReference(Guid.NewGuid().ToString());

        //ServicePointManager.UseNagleAlgorithm = false; // If you change the nagle algorithm here, the performance will be okay.
        queue.CreateIfNotExist();
        ServicePointManager.UseNagleAlgorithm = false; // TOO LATE, the queue is already created without 'nagle'
        var w = new Stopwatch();
        w.Start();
        for (int i = 0; i < 50;i++ )
        {
            Console.WriteLine("nr {0}",i);
            queue.AddMessage(new CloudQueueMessage("hello "+i));    
        }

        w.Stop();
        Console.WriteLine("elapsed: {0}", w.ElapsedMilliseconds);
        queue.Delete();

The suggested solution from Sandrino to configure the ServicePointManager using the app.config file has the advantage that the ServicePointManager is initialized when the application starts up, so you don’t have to worry about time dependencies.

Answer 1

I answered a similar question a few days ago: How to achive more 10 inserts per second with azure storage tables.

For adding 1000 items in table storage it took over 3 minutes, and with the changes I described in my answer it dropped to 4 seconds (250 requests/sec). In the end, table storage and storage queues aren’t all that different. The backend is the same, data is simply stored in a different way. And both table storage and queues are exposed through a REST API, so if you improve the way you handle your requests, you’ll get a better performance.

The most important changes:

• expect100Continue: false
• useNagleAlgorithm: false (you’re already doing this)
• Parallel requests combined with connectionManagement/maxconnection