I’m currently writing a method which, in order to reduce transaction costs, inserts entities into Azure Table Services in batches of 100.
The entities inserted are immutable. I.e. if the PartitionKey and the RowKey are identical between two entitiy-object it means that all other properties are identical as well. Therefore I will never need to make updates to this table, because after insertion the entity will stay the same.
The problem I’m facing is that within this batch I cannot be 100% sure that none of the entities haven’t been inserted before. Therefore, if I’m just inserting the entities as normal (through AddObject and SaveChanges) the transaction might fail with the error that the entity cannot be added, because it already exists. Since the transaction is atomic, this means that none of the entitites are added.
I cannot check for existence of each entity in the transaction, because that would be too costly (one query per entity costs too much and a single query using OR results in a table scan across the entire table). I therefore need some way to add an object if it doesn’t exist and ignore it or update it, if it exists (ignore or update would be the same for me since the entitites are immutable).
How do I achieve this in Azure Table Services?
According to this article there is currently not a way to “upsert” an entity if it already exists. The current recommendation is to process the request using multiple worker roles and multiple threads checking for the existence of each entity.