Question

I am developing a library that uses one or more helper executable in the course of doing business. My current implementation requires that the user have the helper executable installed on the system in a known location. For the library to function properly the helper app must be in the correct location and be the correct version.

I would like to removed the requirement that the system be configured in the above manner.

Is there a way to bundle the helper executable in the library such that it could be unpacked at runtime, installed in a temporary directory, and used for the duration of one run? At the end of the run the temporary executable could be removed.

I have considered automatically generating an file containing an unsigned char array that contains the text of the executable. This would be done at compile time as part of the build process. At runtime this string would be written to a file thus creating the executable.

Would it be possible to do such a task without writing the executable to a disk (perhaps some sort of RAM disk)? I could envision certain virus scanners and other security software objecting to such an operation. Are there other concerns I should be worried about?

The library is being developed in C/C++ for cross platform use on Windows and Linux.

Answer 1

You can use xxd to convert a binary file to a C header file.

$ echo -en "\001\002\005" > x.binary

$ xxd -i x.binary 
unsigned char x_binary[] = {
  0x01, 0x02, 0x05
};
unsigned int x_binary_len = 3;

xxd is pretty standard on *nix systems, and it’s available on Windows with Cygwin or MinGW, or Vim includes it in the standard installer as well. This is an extremely cross-platform way to include binary data into compiled code.

Another approach is to use objcopy to append data on to the end of an executable — IIRC you can obtain objcopy and use it for PEs on Windows.

One approach I like a little better than that is to just append raw data straight onto the end of your executable file. In the executable, you seek to the end of the file, and read in a number, indicating the size of the attached binary data. Then you seek backwards that many bytes, and fread that data and copy it out to the filesystem, where you could treat it as an executable file. This is incidentally the way that many, if not all, self-extracting executables are created.

If you append the binary data, it works with both Windows PE files and *nix ELF files — neither of them read past the “limit” of the executable.

Of course, if you need to append multiple files, you can either append a tar/zip file to your exe, or you’ll need a slightly more advance data structure to read what’s been appended.

You’ll also probably want to UPX your executables before you append them.

You might also be interested in the LZO library, which is reportedly one of the fastest-decompressing compression libraries. They have a MiniLZO library that you can use for a very lightweight decompressor. However, the LZO libraries are GPL licensed, so that might mean you can’t include it in your source code unless your code is GPLed as well. On the other hand, there are commercial licenses available.