In my application, I have a process which forks off a child, say child1, and this child process writes a huge binary file on the disk and exits. The parent process then forks off another child process, child2, which reads in this huge file to do further processing.
The file dumping and re-loading is making my application slow and I’m thinking of possible
ways of avoiding disk I/O completely. Possible ways I have identified are ram-disk or tmpfs.
Can I somehow implement ram-disk or tmpfs from within my application? Or is there any other
way by which I can avoid disk I/O completely and send data across processes reliably.
If the two sub-processes do not run at the same time pipes or sockets won’t work for you – their buffers would be too small for the ‘huge binary file’ and the first process will block waiting for anything for reading the data.
In such case you rather need some kind of shared memory. You can use the SysV IPC shared memory API, POSIX shared memory API (which internally uses tmpfs on recent Linux) or use files on a tmpfs (usually mounted on /dev/shm, sometimes on /tmp) file system directly.