Question

In my Table storage, there are 10.000 elements per partition. Now I would like to load a whole partition into memory. However, this is taking very long. I was wondering if I am doing something wrong, or if there is a way to do this faster. Here is my code:

        public List<T> GetPartition<T>(string partitionKey) where T : TableServiceEntity
        {
            CloudTableQuery<T> partitionQuery = (from e in _context.CreateQuery<T>(TableName)
                                                 where e.PartitionKey == partitionKey
                                                 select e).AsTableServiceQuery<T>();

            return partitionQuery.ToList();
        }

Is this the way it is supposed to be done or is their anything equivalent to the batch insertion for getting elements out of the table again?

Thanks a lot,
Christian

EDIT

We have all the data also available in blob storage. That means, one partition is serialized completely as byte[] and saved in a blob. When I retrieve that from blob storage and afterwards deserialize it, it is way faster than taking it from the table. Almost 10 times faster! How can this be?

Answer 1

In your case I think turning off change tracking could make a difference:

context.MergeOption = MergeOption.NoTracking;

Take a look on MSDN for other possible improvements: .NET and ADO.NET Data Service Performance Tips for Windows Azure Tables

Edit: To answer your question why a big file in blob storage is faster, you have to know that the max amount of records you can get in a single request is 1000 items. This means, to fetch 10.000 items you’ll need to do 10 requests instead of 1 single request on blob storage. Also, when working with blob storage you don’t go through WCF Data Services which can also have a big impact.