Question

I’m looking using to transfer an accept()ed socket between processes using sendmsg(). In short, I’m trying to build a simple load balancer that can deal with a large number of connections without having to buffer the stream data.

Is this a good idea when dealing with a large number (let’s say hundreds) of concurrent TCP connections? If it matters, my system is Gentoo Linux

Answer 1

Until someone does a benchmark and establishes how “hard” it is to send a file descriptor, this remains speculation (someone might pop up: “Hey, sending the descriptor like that is dirt-cheap”). But here goes.

You will (likely, read above) be better off if you just use threads. You can have the following workflow:

• Start a pool of threads that just wait around for work. Alternatively you can just spawn a new thread when a request arrives (it’s cheaper than you think)
• Use epoll(7) to wait for traffic (wait for connections + interesting traffic)
• When interesting traffic arrives you can just dispatch a “job” to one of the threads.

Now, this does circumvent the whole descriptor sending part. So what’s the catch ? The catch is that if one of the threads crashes, the whole process crashes. So it is up to you to benchmark and decide what’s best for your server.

Personally I would do it the way I outlined it above. Another point: if the workers are children of the process doing the accept, sending the descriptor is unnecessary.