Question

I’m using AppEngine to store some pickled python objects in my app. I want to serve these to the user directly, and I’m simply using the X-AppEngine-Blobkey header to serve the files to the user with a file.pickle.gz filename. However, when I try to extract these on my computer (Mac OS) using a simple double click, the files are turned into file.pickle.gz.cpgz.

I thought it was my browser being sneaky and extracting them, but I don’t think so, since

pickle.load('file.pickle.gz')

Doesn’t work, and neither does

pickle.load('file.pickle.gz.cpgz') 

To store the files, I use:

    blobfile = files.blobstore.create(mime_type='application/gzip')
    with files.open(blobfile, 'a') as f:
        gz = gzip.GzipFile(fileobj=f,mode='wb')
        gz.write(my_pickled_object)
        gz.close()
    files.finalize(blobfile)

I think I’m not understanding the way gzips work. Can someone explain?

Answer 1

Are you sure file.pickle.gz.cpgz is the result of your double-clicking on the file.pickle.gz file you downloaded? Usually “.cpgz” is a different kind of archive file.

I can get the code you posted to work in a development server without significant changes. Here’s the code, if it helps:

#!/usr/bin/env python

from __future__ import with_statement
import gzip
import pickle
from google.appengine.api import files
from google.appengine.api import memcache
from google.appengine.ext import blobstore
from google.appengine.ext import webapp
from google.appengine.ext.webapp import blobstore_handlers
from google.appengine.ext.webapp import util

class MainHandler(webapp.RequestHandler):
    def get(self):
        self.response.out.write('Hello world! <a href="/make">make</a> <a href="/get">get</a>')

class MakeFileHandler(webapp.RequestHandler):
    def get(self):
        data = pickle.dumps({'a':1, 'b':True, 'c':None})

        blobfile = files.blobstore.create(mime_type='application/gzip')
        with files.open(blobfile, 'a') as f:
            gz = gzip.GzipFile(fileobj=f,mode='wb')
            gz.write(data)
            gz.close()
        files.finalize(blobfile)
        memcache.set('filekey', files.blobstore.get_blob_key(blobfile))
        self.redirect('/')

class GetFileHandler(blobstore_handlers.BlobstoreDownloadHandler):
    def get(self):
        blobkey = memcache.get('filekey')
        if blobkey:
            self.send_blob(blobkey)
        else:
            self.response.out.write('No data key set <a href="/">back</a>')

def main():
    application = webapp.WSGIApplication([('/', MainHandler),
                                          ('/make', MakeFileHandler),
                                          ('/get', GetFileHandler)],
                                         debug=True)
    util.run_wsgi_app(application)

if __name__ == '__main__':
    main()

Click on “make”, then click on “get”. A file named “get.gz” is downloaded to your ~/Downloads/ folder (at least in Chrome). Double-click on it to produce a file named “get”. Then:

% python
>>> import pickle
>>> pickle.load(open('get'))
{'a': 1, 'c': None, 'b': True}