Question

I am writing a service that will connect to a single socket then read the data which will be continuous and at times plentiful. The data I receive should then be handed off to another thread (in the service) to process, the reason is that the processing might take a while and I don’t want to block the receiving of the socket.

I was going to do this using the Begin or Async receive methods to process the data rather than having a thread running reading a concurrent queue or something. I need this to scale to possibly large amounts of data coming over the socket. Any suggestions?

Answer 1

I’m not sure how using asynchronous receive is going to help with the processing. As I understand it, you’ll have one thread that connects to the socket and reads a continuous stream of data. I assume that the thread can break that continuous stream into records of some kind, or at least blocks of manageable size. You then have to make that data available to one or more processing threads.

I would strongly suggest that you do use a concurrent queue (actually a BlockingCollection) to communicate between the thread that’s reading the data and the thread or threads that will be doing the processing. There are several reasons I suggest using BlockingCollection:

• It’s is easy to use – no explicit locking code
• It does non-busy waits
• It handles N readers and N writers efficiently

Yours is a pretty typical producer/consumer application. The producer (the thread reading the socket) gets data, places it into the queue, and then one or more consumers (processing threads) can read and process the data. I can’t think of a simpler, more effective, or more efficient way to do this in .NET than with a BlockingCollection.