I am considering beginning work on a library for Linux that would provide a virtual file system to application developers where the files would be stored in an archive, and each file within the archive would be individually compressed so that retrieval of a single file is a very straightforward task for the developer, for the CPU, and the hard drive. (No complicated API, no need to uncompress gigs of data, just the data that’s relevant, and retrieval of only relevant data rather than the whole archive)
I’ve used popen to retrieve the stdout of a command before using C++ here on Linux, but I don’t know how to pipe data in and get data out, and some bzip2 specific tips would be nice. I wrote something similar to this years ago, but it included a huffman compression library as a dll, rather than having to pipe data and use a standard tool. (that was back in my Windows days.)
bzip2 has a library interface — that will probably be easier for you than invoking a subprocess.
I recommend you also have a look at the GIO library, which is already a “virtual file system for application developers”; it might be a lot less work to extend that to do what you want, than to write a library VFS from scratch.