Question

What is the best way to implement C++/Java IPC for the following situation?

(Someone recently asked a similar question, but my requirements are more specific)

1. I have two programs — one written in C++, the other in Java — that need to communicate with each other. Both are running on the same machine.

2. The programs send messages to each other. Messages are typically short (less than a few hundred bytes), but could potentially be 100KB or more in size.

3. Messages do not need to be acknowledged (i.e., not a request/response model like HTTP). For example, the C++ program sends a message to the Java program, and the Java program may reply by sending a message to the C++ program at a later time — and vice versa.

4. An ideal solution would have a) very low latency, b) no security hassles (user does not have to authorize ports to be opened etc.) and c) will be platform-agnostic.

My first thought was using sockets — each program would act as a server to the other. Sockets have more overhead than other forms of IPC, and I don’t know how the server would inform the client of the port number if I let the system auto-assign port numbers. I’ve also considered named pipes, but they are not supported (at least not consistently) across different platforms. JNI looks like an option, but can it cross process boundaries?

Any suggestions?

Thanks!

FOLLOW-UP QUESTIONS

1. If I go with sockets, would I need to open two sockets to allow for asynchronous communication as described above?

Answer 1

I’d suggest you to use TCP sockets.

The actual overhead of TCP sockets, as of my experience, is very very low compared to the other tasks’ workload of the applications, at least the ones I use to develop. I mean, sometimes even if sockets’ latency is twice as the latency of other IPC mechanisms, in the overall workflow they have very little impact. And it saves you the hassle of making IPC between a Java application and a C++ one, that will eventually require you to use a specific Java library that uses JNI, with the overhead of JNI and the one of the library itself.

I’ve actually measured, in my Java applications, that Garbage Collector impact is far more important than the latency caused by “loopback” TCP sockets.

Moreover, TCP sockets are more scalable (and portable!) than traditional IPC. What if in the future you’ll have to run the client and the server on different machines? In the ‘TCP sockets’ scenario, you’ll have to do a 5-minute hack, in the ‘traditional IPC’ scenario, you’ll have to rewrite the whole IPC stuff.

However, what is the general workflow of your application?

Even if the acknowledgement is not required, I’d suggest to use TCP (and not UDP) to avoid unsorted delivery (which leads to pain in the ass when it comes to rearrange the stuff you received – some of your messages are 100KB and this doesn’t fit in a UDP packet).

In reply to your last question, for the server to inform the client about the port, you can just make the server launch the client with a specific ‘port’ command line parameter, or make the server save a small file under /tmp (or another temporary directory) with the port number written inside.