Question

I’m using Nick Johnson’s Bulk Update library on google appengine (http://blog.notdot.net/2010/03/Announcing-a-robust-datastore-bulk-update-utility-for-App-Engine). It works wonderfully for other tasks, but for some reason with the following code:

 from google.appengine.ext import db
 from myapp.main.models import Story, Comment
 import bulkupdate

 class Migrate(bulkupdate.BulkUpdater):
     DELETE_COMPLETED_JOBS_DELAY = 0
     DELETE_FAILED_JOBS = False
     PUT_BATCH_SIZE = 1
     DELETE_BATCH_SIZE = 1
     MAX_EXECUTION_TIME = 10

     def get_query(self):
         return Story.all().filter("hidden", False).filter("visible", True)

     def handle_entity(self, entity):
         comments = entity.comment_set
         for comment in comments:
             s = Story()
             s.parent_story = comment.story
             s.user = comment.user
             s.text = comment.text
             s.submitted = comment.submitted
             self.put(s)

 job = Migrate()
 job.start()

I get the following error in my logs:

Permanent failure attempting to execute task
Traceback (most recent call last):
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 258, in post
    run(self.request.body)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 122, in run
    raise PermanentTaskFailure(e)
PermanentTaskFailure: 'module' object has no attribute 'Migrate'

It seems quite bizarre to me. Clearly that class is right above the job, they’re in the same file and clearly the job.start is being called. Why can’t it see my Migrate class?

EDIT: I added this update job in a newer version of the code, which isn’t the default. I invoke the job with the correct URL (http://version.myapp.appspot.com/migrate). Is it possible this is related to the fact that it isn’t the ‘default’ version served by App Engine?

Answer 1

It seems likely that your declaration of the ‘Migrate’ class is in the handler script (Eg, the one directly invoked by app.yaml). A limitation of deferred is that you can’t use it to call functions defined in the handler module.

Incidentally, my bulk update library is deprecated in favor of App Engine’s mapreduce support; you should probably use that instead.