Question

Currently, I have a dictionary that has a number as the key and a Class as a value. I can access the attributes of that Class like so:

dictionary[str(instantiated_class_id_number)].attribute1

Due to memory issues, I want to use the shelve module. I am wondering if doing so is plausible. Does a shelve dictionary act the exact same as a standard dictionary? If not, how does it differ?

Answer 1

Shelve doesn’t act extactly the same as dictionary, notably when modifying objects that are already in the dictionary.

The difference is that when you add a class to a dictionary a reference is stored, but shelve keeps a pickled (serialized) copy of the object. If you then modify the object you will
modify the in-memory copy, but not the pickled version. That can be handled (mostly) transparently by shelf.sync() and shelf.close(),
which write out entries. Making all that work does require tracking all retrieved objects which haven’t been written back yet so you do have to call shelf.sync() to clear the cache.

The problem with shelf.sync() clearing the cache is that you can keep a reference to the object and modify it again.

This code doesn’t work as expected with a shelf, but will work with a dictionary:

s["foo"] = MyClass()
s["foo"].X = 8 
p = s["foo"] # store a reference to the object
p.X = 9 # update the reference
s.sync() # flushes the cache
p.X = 0
print "value in memory: %d" % p.X # prints 0
print "value in shelf: %d" % s["foo"].X # prints 9

Sync flushes the cache so the modified ‘p’ object is lost from the cache so it isn’t written back.