Question

I have about 900K entities of a model in python GAE that I would like to export to a CSV file for offline testing. I can use the appcfg.py download_data option, but in this case I don’t want to backup to local machine. I’d like a faster way to create the file in GAE, save it to Google Storage or elsewhere, and download it later from multiple machines.

I’m assuming that I will need to do this in a task since it will likely take more than 30 seconds for the operation to complete.

class MyModel(db.model):
  foo = db.StringProperty(required=True)
  bar = db.StringProperty(required=True)

def backup_mymodel_to_file():
  #What to do here?

Answer 1

Your best option will be to use map reduce library to export the relevant data to the blobstore, then upload the completed file to Google Storage.

Note that integration between Google Storage and App Engine is a work in progress.