I am developing a client application that manages one socket. I am using IOCP to manage asynchronous I/O.
This is a quote from a networking programming book:
All overlapped operations are guaranteed to be executed in the order that the application issued them. However, the completion notifications returned from a completion port are not guaranteed to be in that same order. That is, if an application posts two overlapped WSARecv operations, one with a 10 KB buffer and the next with a 12 KB buffer, the 10 KB buffer is filled first, followed by the 12 KB buffer. The application’s worker thread may receive notification from GetQueuedCompletionStatus for the 12 KB WSARecv before the completion event for the 10 KB operation. Of course, this is only an issue when multiple operations are posted on a socket.
So how should I go about dealing with this case?
Simplest solution is, to not have more than one outstanding read or write. So if a completion is notified it’s clear, that all data was written to the socket. If, while waiting for IO to be completed, there is a need to write more data to a socket, simply buffer that data and write it, when the IO gets completed.
If you have more than one outstanding IO to a socket, you have to order them on the read side. On the write side, you have to do some book keeping to know which write was the last and thus, when data was written completely.
On the other hand, just by writing data to successfully to the socket doesn’t mean, that data was correctly received nor processed on the other side, so there have to be some kind of protocol. If so, you can simply rely on that protocol and simply look for error notifications.