Question

Here is the scenario:

There are clients sending requests to a server (it will be sockets or wcf server, that is not important).

Server will keep an open duplex channel and will use it to send an answer (serialized data) to a client. Server will process requests and involves query generation (basing on parameters from a request) and execution against data sources of various types (sql server, file system, analysis services server – olaps, offline cubes and so on…). So heavy IO-bound tasks – definitely often long running.

Performance is important, consider hundreds or maybe thousands of requests at the same time. It must be scalable.

I have never used TPL nor written a asynchronous server. But I’ve read a lot for a few days and… still can’t wrap my head around it.

1. Is TPL (4.0, not 4.5) a good choice here?
2. Should I create tpl Task for every request that comes to a server? (for async processing)
3. Should I create those Tasks with LongRunning option? (so no ThreadPool involved)
4. Should I implement any queue mechanism for requests? How?
5. Should I chain all parts of a request processing (a. query generation b. query execution against data source) with separate tasks (continuations) or is it ok to use single a task for both a. and b.?
6. Should I use .FromAsync task generation for query executions? Or standard .StartNew is enough?
7. What are other important areas I should watch for, given those above requirements?

Answer 1

There has been a lot of discussions on the topic… See https://stackoverflow.com/a/908766/1876226 for example. There Chris Mullins’ posts are mentioned, that are lost. I had a link to some web-archive that has a cached version of one the post, will look for it.

UPDATE:
Found a note of Chris Mullins’s post:

We did quite a bit of scalability testing in some labs using HUGE IA64
servers. The results of that, along with quite a bit of the best
practices that we came up with are detailed at:
http://www.coversant.net/Default.aspx?tabid=88&EntryID=10

see archived copy here: http://nleghari.wetpaint.com/page/Windows+Sockets+and+Threading%3A+How+well+does+it+scale%3F

I am still wrapping my head around the similar application I’m developing. Points learned so far that come to my mind:

• Socket asynchronous operations (Begin*, End*, *Async) use IOCP;
• TPL is a straightforward and robust way of implementing multithreading;
• If you want async – then you should create a Task for request processing. Maybe with the exception of requests that are “known” to be lightweight;
• If you create LongRunning Tasks with hundreds or maybe thousands of requests at the same time, then you may get that number of threads putting your system to it’s knees. Use LongRunning when you need to. Use custom TaskScheduler for concurrency control;
• Queues will inevitably show up in your code: either for batching jobs to avoid exsessive locking and context switch or in a producer-consumer scenarios;
• Some resources (SQL Server, file system…) have async API that also uses IOCP. Some resources don’t. In the former case .FromAsync will effectively use IOCP. But note that IOCP thread pool is also not limitless: it may become exhasted by DB queires and Network servicing;
• If possible – avoid DB access at all via implementing in-app caching
  (.NET MemoryCache or custom) or batching DB requests that do not
  require a response (to make a bulk insert, for example);
• There is a ParallelExtensionsExtras open source library that
  partially became a part of .NET 4.5 Async. So you can have neet TPL
  extensions in .NET 4.0. See tour A Tour of
  ParallelExtensionsExtras

There are a lot of naive and buggy implementations of c# asynchronous socket server. Read, but recheck.