Question

Problem:
I have large number of big messages to serialize and send over network.
I would like to maximize performance, so I’m thinking about creating multiple threads for message serialization and one thread for sending the data. Idea is to dynamically determine number of threads for serialization based on network performance. If data is sent fast and serialization is bottleneck, add more threads to boost serialization. If network is slow, use less threads and stop completely if send buffer is full.

Any ideas how to do that?

What would be the best algorithm to decide if more or less threads are needed?

How to correctly concatenate serialization results from multiple threads?

Please answer to any of this questions? Thanks

Answer 1

It can be treated as a Producer/Consumer problem, In Fx4 you can use a BlockingCollection.

But frankly I would expect the (network) I/O to be the bottleneck, not the serialization. You will have to measure.