Question

Hoping someone can help me design this correctly.

In my TCP code, I have a SendMessage() function that tries to write to the wire. I am trying to design the call so that it moves to a producer/consumer model if a lot of concurrent requests happen, but at the same time, stays single-threaded if there are no concurrent requests (for maximum performance).

I’m struggling on how to design this without race conditions because there is no way to move locks between threads.

What I have so far is something like (pseudo-coded):

SendMessage(msg) {

    if(Monitor.TryEnter(wirelock,200)) {

       try{
          sendBytes(msg);
       }
       finally {
          Monitor.Exit...
       }
   }
   else {

     _SomeThreadSafeQueue.add(msg)

     Monitor.TryEnter(consumerlock,..

          Task.Factory.New(ConsumerThreadMethod....
    }
}


ConsumerThreadMethod() {

   lock (wirelock) {

         while(therearemessagesinthequeue)

               sendBytes...

   }

}

Any obvious race conditions?

EDIT: Found a flaw in the last one. How about this instead?

SendMessage(msg) {

    if(Monitor.TryEnter(wirelock)) {

       try{
          sendBytes(msg);
       }
       finally {
          Monitor.Exit...
       }
   }
   else {

         _SomeThreadSafeQueue.add(msg)

        if (Interlocked.Increment(ref _threadcounter) == 1)
        {
            Task.Factory.StartNew(() => ConsumerThreadMethod());

        }
        else
        {
            Interlocked.Decrement(ref _threadcounter);

        }      
    }
}


ConsumerThreadMethod() {

     while(therearemessagesinthequeue)
           lock (wirelock) {
               sendBytes...
            }
      }

     Interlocked.Decrement(ref _threadcounter);
}

So basically using the interlocked counter as a way to only ever spawn one thread (if necessary)

Answer 1

No obvious races but TryEnter is a cause for some serious idle time. I actually think that using a consumer thread all the time is the best solution. If there is little to do, the overhead will be really small (the consumer thread will be asleep when not working, if designed correctly).

Now you create a new task for each sent message, resulting in huge contention on the lock, since you are using a while loop in the consumer thread.

EDIT: Since you are using non-blocking sockets, a single consumer thread should be enough to handle all send requests. The throughput of a single thread is higher than your network. If you have more consumers it’s hard to make sure that no two consumer threads send on the same socket, without serializing everything using a mutex. I don’t think switching between single-threaded and multi-threaded is a good idea.

Your current “multithreaded” solution does not give you any performance gain since all work is protected using the same mutex. It will be as slow, or slower, than a single thread.