I have a Netty client sending messages asynchronously to a TCP Netty server.
In order to preserve the order the server ChannelPipeline has and ExecutionHandler coupled with an OrderedMemoryAwareThreadPoolExecutor.
My understanding is messages sent to the server are queued up in the channel. If the server dies while messages queued up in the channel I need to prevent message loss.
My current solution is to add each message to a queue on the Client side and only remove them from the queue when I receive an Ack message from the server for each message. What do you think?
What you’re suggesting is fairly common and is sometimes known as a send window. I’ve used similar techniques before, although I stored my sent messages in a map because the server could theoretically process and acknowledge them out of order.
TCP uses similar techniques, as do protocols like SMPP and even JMS providers that allow asynchronous sends (hornetQ for example). HTTP pipelining is similar, although that does require requests to be processed and acknowledged in order.
One thing you have to consider is what the server does with retransmitted messages. It may have already processed the original message but the ack was lost before it reached your client. For example, if the messages are requesting a payment of some kind, you don’t want to charge the person twice because the ack got lost.