The Windows Azure Storage Abstractions and their Scalability Targets blog post indicates there is a 5,000 entities/second transaction limit for a single storage account, and there is a 500 entities/second limit for a single table partition. And to meet the first limit one should used multiple accounts, and for the partition limit one should design their partitions carefully.
I’d like to ask for others who have experience on the 5000 limit to a single storage account. Right now, I’m designing a community of blogs/wikis and say one day the site becomes popular and attracts a lot of traffic. Should I split the user related tables to one storage account and blog related tables to another account and yet wiki related tables to another to prevent this limit right now? Or should I add more accounts as there is a need, by the way is there a way to transfer azure storage tables from one account to another? The article says when you hit that limit you will get “503 server busy” responses, is there a way to know the limit is getting close so I could something in advance without accully resulting 503 errors?
I haven’t hit the account limit overall, but I have hit the limit for number of transactions on a Queue by trying to set the number of worker roles reading from that queue to a ridiculous level.
As far as I know there is no “you’re about to hit the limit” warning. The first time you know that you’ve hit the limit is you get the 503 error.
With transferring data from one account to another, there is no built in functionality that will do it for you. You either have to roll your own solution to read through every row in the source table and write it to the destination table, or use something like the Cerebrata Cloud Storage Studio which allows you to download and upload the contents of tables or their CMDLTS which let you do the same thing, but are cheaper/free.
If you’re just starting out and you have logical ways of partitioning the data across storage accounts and it doesn’t make the code too complicated, then do it. But I wouldn’t worry about it too much at this stage. Chances are if your site does become popular and you start hitting the transaction limit, it will likely come from an area that you hadn’t expected or may come from too many transactions to just one table. As you said this was for a community of blogs, the area that’s likely to get the most transactions is where ever you store comments. If you get more than 5000 transactions a second against your comments table you may need to partition the comments across multiple storage accounts. Of course if the blogs are that popular, chances are you’ll have other problems to deal with as well.