Question

and thanks for taking a look at the question.

The background
I have several machines that continuously spawn multiple (up to 300) PHP console scripts in a very short time frame. These scripts run quickly (less than a second) and then exit. All of these scripts need read only access to a large trie structure which would be very expensive to load into memory each time each one of the scripts runs. The server runs Linux.

My solution
Create a C daemon that keeps the trie structure in memory and receives requests from the PHP clients. It would receive a request from every one of the PHP clients, perform the lookup on the memory structure and respond with the answer, saving the PHP scripts from doing that work. Both requests and responses are short strings (no longer than 20 characters)

My problem
I am very new to C daemons and inter process communication. After much research, I have narrowed the choices down to Message Queues and Unix domain sockets. Message Queues seem adequate because I think (I may be wrong) that they queue up all of the requests for the daemon to answer them serially. Unix domain sockets seem to be easier to use, though. However, I have various questions I have not been able to find answers to:

1. How can a PHP script send and receive messages or use a UNIX socket to communicate with the daemon? Conversely how does the C daemon keep track of which PHP process it has to send a reply to?
2. Most examples of daemons I have seen use an infinite while loop with a sleep condition inside. My daemon needs to service many connections that can come at any time, and response latency is critical. How would the daemon react if the PHP script sends a request while it is sleeping? I have read about poll and epoll, would this be the correct way to wait for a received message?
3. Each PHP process will always send one request, and then will wait to receive a response. I need to make sure that if the daemon is down / unavailable, the PHP process will wait for a response for a set maximum time, and if no answer is received will continue regardless instead of hanging. Can this be done?

The actual lookup of the data structure is very fast, I don’t need any complex multi-threading or similar solution, as I believe handling the requests in a FIFO manner will be enough. I also need to keep it simple stupid, as this is a mission critical service, and I am fairly new to this type of program. (I know, but I really have no way around this, and the learning experience will be great)

I would really appreciate code snippets that shine some light into the specific questions that I have. Links to guides and pointers that will further my understanding into this murky world of low level IPC are also welcome.

Thanks for your help!

---

Update

Knowing much more now than I did at the time of asking this question, I just wanted to point out to anybody interested that both the Thrift framework and ZeroMQ do a fantastic job of abstracting away the hard, socket-level programming. Thrift even gives you the scaffolding for the server for free!

In fact, instead of going to all the hard work of building a network server, consider just writing you applications server code using a good asynchronous server that has already solved the problem for you. Of course servers that use asynchronous IO are great for network applications that don’t require intensive CPU processing (or else the event loop blocks).

Examples for python: Twisted, gevent. I prefer gevent, and I don’t include tornado because it is focused on the HTTP server side.

Examples for Ruby: EventMachine

Of course, Node.js is basically the default choice for an async server nowadays.

If you want to go deeper, read the C10k Problem, and Unix Network Programing.

Answer 1

I suspect Thrift is what you want. You’d have to write a little glue code to do PHP <-thrift-> C++ <-> C, but that would probably be more robust than rolling your own.