Question

What is the most efficient way of implementing queues to be read by another thread/process?

I’m thinking of using a basic MySQL table with polling on sleep. This sounds to be the most scalable (it doesn’t even have to be on the same server) but might potentially result in too many queries to the DB.

Answer 1

You have several options, and it really depends on what you are trying to get the system to do.

• fork child processes, and interface using connections their stdin/stdout pipes.
• create a named pipe on the file system, like /tmp/mysql.sock. This is basically using sockets to communicate cross process.
• Setup a message broker. I’d recommend giving ActiveMQ a try, and using the Stomp client for Perl. This is probably your most scalable solution.