Question

During instantiation of my class, I initialize some fields that are not picklable. Thus, in order to be able to (un)pickle my classes correctly, I would like my init method to be called on unpickling. This is, it seems, the way it worked with old-style classes.

With new style classes, I need to use __new__ and __getnewargs__. Here is what I do:

import cPickle


class Blubb(object):

  def __init__(self, value):
    self.value = value


class Bla(Blubb):

  def __new__(cls, value):
    instance = super(Bla, cls).__new__(cls)
    instance.__init__(value)
    return instance

  def __getnewargs__(self):
    return self.value,

  def __getstate__(self):
    return {}

  def __setstate__(self, dct):
    pass

x = Bla(2)
print x.value
pickled = cPickle.dumps(x, 2)
x_ = cPickle.loads(pickled)
assert x_.value == 2

This would be fine, if not for the fact that obj = C.__new__(C, *args). There is now **kwargs. So I am restricted to non keyword arguments in my __new__ and __init__ methods.

Does anyone know of a way to solve this? This is really unconvenient.

Answer 1

The pickle protocol 2 wants to call cls.__new__(cls, *args) by default, but there is a way around this. If you use __reduce__ you can return a function which will map your arguments to __new__. I was able to modify your example to get **kwargs to work:

import cPickle

class Blubb(object):

    def __init__(self, value, foo=None, bar=None):
        self.value = value
        self.foo = foo
        self.bar = bar

def _new_Bla(cls, value, kw):
    "A function to map kwargs into cls.__new__"
    return cls.__new__(cls, value, **kw)

class Bla(Blubb):

    def __new__(cls, value, **kw):
        instance = super(Bla, cls).__new__(cls)
        instance.__init__(value, **kw)
        return instance

    def __reduce__(self):
        kwargs = {'foo': self.foo, 'bar': self.bar}
        return _new_Bla, (self.__class__, self.value, kwargs), None

x = Bla(2, bar=[1, 2, 3])
pickled = cPickle.dumps(x, 2)
y = cPickle.loads(pickled)
assert y.value == 2
assert y.bar == [1, 2, 3]