Question

I’m working to port some relatively simple network code from unix to Windows.

In brief, there’s a single network thread which manages all network traffic. This single thread sits inside poll() most of the time (I’ve already converted this to select(), on Windows), the network thread only waking up when there’s incoming network data. When a client command arrives, the network thread fires the command off to a worker thread to actually perform work on the back-end, and then goes back to listening for more incoming commands. Commands are asynchronous — multiple commands can come in on one socket, one right after the next. At any one time, there can be dozens of outstanding commands on a single connection, all being worked on in various worker threads.

The problem is that when the worker thread wants to send response data, the network thread is probably asleep inside poll() at the time that the worker thread is trying to send its result. The network thread has to wait for poll() to time out (or another packet to be received) before it will notice that there’s new outbound data queued up to be sent.

Under unix, I deal with this by including a pipe amongst the descriptors being watched by poll(), and when a worker thread has outbound data to send, it also writes a byte of data into the pipe to wake up the network thread. But WinSock only seems to support waiting on sockets, so that approach isn’t going to work for me in Windows. Is there any way for me to salvage this architecture on Windows using WinSock (so I can share most of the code between different platforms), or is there no option but to write a custom network server implementation for use on Windows?

Thanks for any advice!

Answer 1

There can only be two possibilities, and neither requires the solution you are proposing:

1) You are already trying to send data on this connection and the OS’s send queue is full. In this case, there is no need to abort out of select or poll. You can’t write on the connection right now anyway, and you’ll exit select or poll automatically as soon as you can.

2) You are not currently trying to send data on this connection. In this case, there is also no need to abort out of select or poll. Just write the data right now in the worker thread. (The socket is non-blocking, right?) If you don’t write all of the data, it would only be because the OS’s send queue is full, in which case there’s usually no rush to write (set a reasonable timeout on the select call).

You really shouldn’t use select on Windows though. It’s an abomination offered only as a gross hack for compatibility. If you plan on “really” supporting Windows, you shouldn’t make it second class. Network I/O is something that really has to be platform-specific.