Question

I looked through several SO-Questions for how to pickle a python object and store it into a database. The information I collected is:

• import pickle or import cpickle. Import the latter, if performance is an issue.
• Assume dict is a python dictionary (or what so ever python object): pickled = pickle.dumps(dict).
• store pickled into a MySQL BLOB Column using what so ever module to communicate with Database.
• Get it out again. And use pickle.loads(pickled) to restore the python dictionary.

I just want to make sure I understood this right. Did I miss something critical? Are there sideeffects? Is it really that easy?

Background-Info: The only thing I want to do, is store Googlegeocoder-Responses, which are nested python dictionarys in my case. I am only using a little part of the response object and I don’t know if I will ever need more of it later on. That’s why I thought of storing the response to save me repetition of some million querys.

Answer 1

It’s really that easy… so long as you don’t need your DB to know anything about the dictionary. If you need any sort of structured data access to the contents of the dictionary, then you’re going to have to get more involved.

Another gotcha might be what you intend to put in the dict. Python’s pickle serialization is quite intelligent and can handle most cases without any need for adding custom support. However, when it doesn’t work, it can be very difficult to understand what’s gone wrong. So if you can, restrict the contents of the dict to Python’s built-in types. If you start adding instances of custom classes, keep them to simple custom classes that don’t do any funny stuff with attribute storage or access. And beware of adding instances of classes or types from add-ons. In general, if you start running into hard-to-understand problems with the pickling or unpickling, look at the non-built-in types in the dict.