Question

I created a test web application to test persist-read-delete of Entities, I created a simple loop to persist an Entity, retrieve and modify it then delete it for 100 times.

At some interval of the loop there’s no problem, however there are intervals that there is an error that Entity already exist and thus can’t be persisted (a custom exception handling I added).

Also at some interval of the loop, the Entity can’t be modified because it does not exist, and finally at some interval the Entity can’t be deleted because it does not exist.

I understand that the loop may be so fast that the operation to the Appengine datastore is not yet complete. Thus causing, errors like Entity does not exist, when trying to access it or the delete operation is not yet finished so creating an Entity with the same ID can’t be created yet and so forth.

However, I want to understand how to handle these kind of situation where concurrent operation is being done with a Entity.

Answer 1

From what I understand you are doings something like the following:

for i in range(0,100):
    ent = My_Entity()    # create and save entity
    db.put(ent)
    ent = db.get(ent.key())   # get, modify and save the entity
    ent.property = 'foo'
    db.put(ent)
    ent.get(ent.key())  # get and delete the entity
    db.delete(my_ent)

with some error checking to make sure you have entities to delete, modify, and you are running into a bunch of errors about finding the entity to delete or modify. As you say, this is because the calls aren’t guaranteed to be executed in order.

However, I want to understand how to handle these kind of situation where concurrent operation is being done with a Entity.

You’re best bet for this is to batch any modifications you are doing for an entity persisting. For example if you are going to be creating/saving/modifying/savings or modifying/saving/deleting where ever possible try to combine these steps (ie create/modify/save or modify/delete). Not only will this avoid the errors you’re seeing but it will also cut down on your RPCs. Following this strategy the above loop would be reduced to…

prop = None
for i in range(0,100):
    prop = 'foo'

Put in other words, for anything that requires setting/deleting that quickly just use a local variable. That’s GAE’s answer for you. After you figure out all the quick stuff you can’t persist that information in an entity.

Other than that there isn’t much you can do. Transactions can help you if you need to make sure a bunch of entities are updated together but won’t help if you’re trying to multiple things to one entity at once.

EDIT: You could also look at the pipelines API.