Question

I am trying to data a data array created in Python3.2, pickle it, and then open it in Python2.7. However, there is some part of the data that Python2.7 is objecting to, even though on a sample of the data it does fine, and I was wondering how to figure out what was going wrong.

So in Python3.2:

import pickle
with open('c:\\test.pickle', mode='wb') as f:
    pickle.dump(t_array, f, 2)

Then, when reading in Python2.7:

import pickle
f = open('c:\\test.pickle', mode='rb')
t_data = pickle.load(f)

The error is:

File "C:\Python27\lib\pickle.py", line 1378, in load
    return Unpickler(file).load()
File "C:\Python27\lib\pickle.py", line 858, in load
    dispatch[key](self)
File "C:\Python27\lib\pickle.py", line 1217, in load_build
    setstate(state)
TypeError: must be char, not unicode

The data is an array of dicts, nested at most two deep, e.g:

{'key3': '3', 'key2': 1.1, 'key1': 1, 'dict': {'dkey2': 2, 'dkey1': 1}}

What’s (probably) going wrong here? Is there any easy way to see what in the original (large) dataset is causing the problem?

Answer 1

Pickle isn’t as good as it might look like. Security vulnerabilities and many issues like these.

A much better idea is to create your own save format, for example using json.