Question

For example, Marshal is still parsing the input data, according to Python source.

.....
case TYPE_FALSE:
    Py_INCREF(Py_False);
    retval = Py_False;
    break;

case TYPE_TRUE:
    Py_INCREF(Py_True);
    retval = Py_True;
    break;

case TYPE_INT:
    retval = PyInt_FromLong(r_long(p));
    break;

case TYPE_INT64:
    retval = r_long64(p);
    break;

case TYPE_LONG:
    retval = r_PyLong(p);
    break;

case TYPE_FLOAT:
.......

Is there any lower layer object manipulation function than “marshal” or “cPickle” in Python?

For example, I already loaded the dumped data to memory, which I just want to type casting like we can do in C/C++, (PyObject *) data_loaded_in_memory;

Edit: If this cannot be done in python directly, any hints about C functions to write that ability would be great.

Answer 1

Don’t think you could just take the memory image of python object, store it, load it back in different interpreter or even the same one later and expect it to make sense. It will not. Integers and floats may be fully contained in the object structure, but string already has a separately-allocated buffer for the data and even a long does.

In another words, cPickle is the lowest possible layer (cPickle is lower level than marshal, because the later maintains compatibility between versions and platforms, which cPickle does not) allowing to store the objects and load them in another interpreter or the same interpreter if they were released from memory in between.