Question

What I need is a class that does async operations in order.

class FooSocket
{
    private Socket _Socket;

    // Message is a class that wraps a byte array.
    public Task<Message> Receive() { /*Bla...*/ };
    public Task<int> Send(Message message) { /*Bla...*/ };
}

If I call Send, Receive and Send in this order, I need to send first and queue the remaining receiving and sending operations until the first receiving is finished.

I tried creating a main task field in the class and follow a MainTask = MainTask.ContinueWith(...) kind of approach. I even wrote a class called Sequencer which does exactly this but it somehow felt wrong, creating nested tasks (using Task.Factory.FromAsync methods) in continuations and stuff.

I also tried to make something like queuing TaskCompletionSource objects and returning their tasks in my Receive/Send methods, checking the queue in an infinite loop on a seperate thread but since I’ll have about 200k of FooSocket instances, a thread for each also felt unwise. If I make it a thread-pool, I come to this “a thread-pool shouldn’t be used for long-running operations” rule.

I feel close but can’t be sure what is the most efficient way to order these jobs.

Answer 1

I would use TPL Dataflow. You can either install it via NuGet or as part of the Async CTP. TPL Dataflow provides a basic BufferBlock<T> type which sounds like just what you need.

If you’re just modeling a socket, then complete the Send tasks when the data is buffered to go out, continuously read into another buffer, and complete the Receive tasks when you read from that buffer. (Note: a “send” operation on a Socket completes when the data is buffered to the OS, not when it goes out on the wire or reaches its destination).

If you’re modeling a higher-level command/response, then you should have one task that represents the entire command/response, as James suggested. I would do both; layering one on top of the other is easy with async/await support. (Note: a “receive” operation on a Socket may complete with a partial receive, so you need message framing).

Your architecture may need some work as well:

I’ll have about 200k of FooSocket instances

That would definitely be a problem (I’m assuming you’re using TCP/IP). There are only ~65K TCP/IP ports of which only ~16K are ephemeral by default, and you have to leave a significant amount of “breathing room” for the OS or it starts misbehaving. I would estimate only ~12K connections are realistic, unless you change the ephemeral range which could (in theory) get you up to ~59K. ~200K is just not possible – unless you change the ephemeral range and have multiple IP addresses with a load balancer.