Question

previously my program used to serialize the whole std::multimap<Participant*, Connection*> after it has been completely populated. and that was simple arc & _connections for both save and restore.

but that needs every connection object to stay in memory. but I don’t need these objects for anything other that serialization. So to minimize memory consumption its decided to serialize std::make_pair(connection->participant(), connection) as soon as they are created. and delete after serialization is done.

expected size of multimap is known before populate starts.

What I want is to manually serialize these pairs such that I don’t need to alter deserialization code which simply arc & _connections;

from boost/serialization/collections_save_imp.hpp I see

boost::serialization::save_construct_data_adl(
    ar, 
    &(*it), 
    boost::serialization::version<BOOST_DEDUCED_TYPENAME Container::value_type>::value
);
ar << boost::serialization::make_nvp("item", *it++);

So should I need to use something like

typedef std::pair<Participant*, Connection*> PairT;

ar <<  BOOST_SERIALIZATION_NVP(expected_size);

if(3 < ar.get_library_version()){// I don't really understand this magic number here
    const unsigned int item_version = boost::serialization::version<PairT>::value;
    ar << BOOST_SERIALIZATION_NVP(item_version);
}

PairT pair = std::make_pair(connection->participant(), connection);
boost::serialization::save_construct_data_adl(
    ar, 
    pair, 
    boost::serialization::version<PairT>::value
);
ar << boost::serialization::make_nvp("item", pair);
delete connection;

I am not sure exactly how it should be done. just making guesses.

Answer 1

I see what you are trying to do but i wouldn’t recommend it because it breaks easily.
If you change the archive to xml for example, it won’t work. If you upgrade to a newer
version of boost, it may break as well and may be difficult to debug. The reason is that the
archive itself may add some extra data in start_save() (the class id for example), then it
does the actual serialization and finishes up with a call to end_save(). Since those methods are
protected, you cannot use them to ‘fake’ the serialization in a portable/stable manner.

You can override (specialize) the multimap serialization but that would only give you access to the
empty container and you’d have to use some trickery (like global variables) to access the connection
objects you want to serialize:

template<class Archive>
inline void save(Archive& ar, const MultimapT& map, const unsigned version) {
    Connection* p = new Connection( global_data->get(i) );
}

You also have to deserialize the same object you’re serializing so the only clean way is to give the archive an object containing the multimap that deals with the serialization. That object can either be the class that owns the map or a dummy. However, that will require some changes to the existing deserialization code:

ar << boost::serialization::make_nvp("map",Dummy(&_source));
// and
ar >> boost::serialization::make_nvp("map",Dummy(&_connections));

The constructor of the dummy will either take a pointer to the object required to generate the connections
(when saving) or a pointer to the multimap (when loading). The dummy’s save() method can then generate,
store and delete Connection objects on the fly while the load() method simply fills the multimap.

When you generate objects on the fly, you must disable object tracking:

BOOST_CLASS_TRACKING(Connection, boost::serialization::track_never)
BOOST_CLASS_TRACKING(Participant, boost::serialization::track_never)

Otherwise the archive detects that the same memory address is serialized repeatedly and will create references to the first object instead of actually storing the data.

Here’s an example of member functions to demonstrate the serialization of an object containing the multimap (dummy or parent class):

// class Foo {

template<typename Archive>
inline void save( Archive& ar, const unsigned version ) const {
    size_t count = expected_count();
    ar << BOOST_SERIALIZATION_NVP(count);
    for( size_t i=0; i<count; ++i ) {
        Connection* connection = make_connection(i);
        PairT pair(connection->participant(), connection);
        ar << boost::serialization::make_nvp("item", pair);
        delete connection;
    }
}

template<typename Archive>
inline void load( Archive& ar, const unsigned version ) {
    size_t count=0;
    ar >> BOOST_SERIALIZATION_NVP(count);
    while( count-- ) {
        PairT pair;
        ar >> boost::serialization::make_nvp("item", pair);
        _connections->insert(pair);
    }
}

friend boost::serialization::access;
BOOST_SERIALIZATION_SPLIT_MEMBER() // split serialize() into save() and load()
// }

References: intrusive/non-intrusive serialization, splitting save and load, object tracking