Question

I am currently implementing an application protocol library relying on TCP/IP for transport (long lasting connection).

I am trying to achieve a nice asynchronous implementation relying on TAP pattern using C#5 async/await constructs, mainly to put into practice the concepts I have only seen in theory up until now.

The client can connect to a remote server and send requests to it.
client receives response from the server as well as requests (full duplex mode).

From the point of view of the client code, the asynchronous call to my library to send a request to the server and receive the associated response is as simple as :

var rsp = await session.SendRequestAsync(req);

From inside my protocol library, I am just buliding the request, converting it to bytes (to be sent on the network stream) and I call WriteAsync on the stream and I then await on a Task created just before sending the request, making use of a TaskCompletionSource object, which is basically waiting for the associated response to be received (and setting the result on the tcs), and then return the response to the client caller.

This part seems fine.

Now the “problem” concerns the part where server is sending requests to the client. There are different type of requests that the server can send to the client.

My protocol library is using an asynchronous loop to listen to the underlying stream (receiving incoming responses or requests from the server).
This loop is reading responses/requests asynchronously on the stream, then in case of a request from the server, it raises an event corresponding to the request type (such as ReceivedRequestTypeA). The client code can subscribe to these events to be notified when a specific request type is received from the server. The event args of these contains all the parameters associated with the request as well as a response object, to be set by the client, which will be asynchronously sent on the stream by library once event handler code is completed.

The code for the asynchronous listen loop is as follow. Please do not mind the while true, not very pretty (cancelation pattern should be used instead), but this is not the point !

private async Task ListenAsync()
{
  while(true) 
  {
    Request req = await ReadRequestAsync();
    await OnReceivedRequest(req);
  }
}

So the loop is calling the asynchronous method ReadRequestAsync which is just reading some bytes asynchronously in the stream until a complete request or response is available.
Then it forwards the request to the asynchronous method OnReceivedRequest which code can be seen below :

private async Task OnReceivedRequest(Request req)
{
   var eventArgs = new ReceivedRequestEventArgs { Req = req };

   if (req is RequestTypeA)
   { ReceivedRequestTypeA(this, eventArgs); }

   [... Other request types ...]

   await SendResponseAsync(eventArgs.Resp);
} 

This asynchronous method raise the appropriate request type event.
The client code is subscribed to this event, so its appropriate event handler method is called … the client code does whatever it needs with the request and then construct a response and set it in the EventArgs object -end of event handler method-. The code resumes in OnReceivedRequest in the library, and the response is sent asynchronously (calling WriteAsync on the underlying stream).

I don’t think this is a good approach, as it can completely block the asynchronous loop in the library if the event handler code on client side is doing a lengthy blocking operation (bye bye fully asynchronous protocol library, you are now becoming somehow synchronous due to client code). The same would happened if I was using an asynchronous task based delegate for events and awaiting on it.

I was thinking that instead of using events, I could have an asynchronous method GetRequestTypeAAsync() which would be implemented using TaskCompletionSource object in library, and the tcs result being set with the request in OnReceivedRequest. And on client code side, instead of subscribing to ReceivedRequestTypeA event, the code would rather consist of a loop arround GetRequestTypeAAsync(). Still as the client code must somehow provide a response to the library to be sent to server, I don’t know how this could work …

My brain is completely fuzzy right now and can’t really think clear. Any suggestion for a nice design will be greatly appreciated.

Thanks !

Answer 1

I’m also working on async/await TCP/IP sockets, and I strongly recommend you take a look at TPL Dataflow. It’s pretty easy to make async-friendly endpoints using two BufferBlocks (one for reads and one for writes).

In my system, the TCP/IP socket wrapper exposes a simple ISourceBlock<ArraySegment<byte>> representing the raw reads. This is then linked to a TransformManyBlock which performs message framing, and from there it can be linked to a TransformBlock which parses the bytes into actual message instances.

This approach works best if you have a RequestType base class from which all your other message types inherit. Then you can have a single receiving task that just (asynchronously) receives RequestType message instances from the end of the dataflow pipeline.