I would like to pre-allocate sufficiently large amount of memory in a program before forking processes and then further allocate/use memory from this pool in the forked processes. I have come across some memory allocators like Bget, Boost etc but not able to understand how to use them.
Is there a simplest one out there which I can use like
poolhandle = poolallocate(pool_size)
Then in forked process use something like
ptr = allocatefromPool(poolhandle,no_of_bytes)
and then if I pass this pointer to another process through some IPC it should be accessible even in that process.
Can you point me in right direction ? If Boost is the way to go can you provide me an example on how to use it ?
Honestly, the easiest way to do this is to use a memory-mapped file. Then you just do seek and write to save things, or seek and read to get things.
Beyond that, there is always the concept of actors where you don’t share any memory at all, just send messages between actors, each of whom is the curator of their own datastore. This is simpler to program, especially with tools like ZeroMQ that make the interprocess and interthread messaging so simple.
If you combine the two ideas then each actor process has an area of the memory mapped file which they own, and you pass messages back and forth to tell them what to read or write.
See this question for more on memory-mapped files as shared memory Posix shared memory vs mapped files
Here is one about IPC which you will need for multiple processes to coordinate actions IPC vs domain sock vs named pipes
If you have large blocks of data to move around in messages then you would probably want to leave the data in place in the memory-mapped file and implement some form of lock-free sharing. There is lots of stuff you can Google using the keyword lock-free in conjunction with “shared memory” or “data structures”.