Question

I have an app that needs to process some packets from a UDP broadcast, and write the result to a SQL database.
During intervals the rate of incoming packets gets quite high, and I find that the underlying buffer overflows, even at 8MB, because of the slow database writing.

To solve this I have come up with two options, cache some of the DB writing,
and/or marshal the db writes onto another sequentially operating thread. (order of db writes must be preserved).

My question is how to marshal the work most effectively, with built-in structures/features of c#?

Should I just have a synchronized queue, put the delegate work items onto it and have a single thread service it in an event loop?

Answer 1

The easiest way is to create a separate thread for the UDP reading.

The thread just receives the packages and puts them into a BlockingCollection.
(Don’t know how you do the packet reading so this will be pseudocode)

BlockingCollection<Packet> _packets = new BlockingCollection<Packet>();
...

while (true) {
    var packet = udp.GetNextPacket();
    _packets.Add(packet);
}

This will ensure you don’t miss any packages because the reading thread is busy.

Then you create a worker thread that does the same thing as your current code but it reads from the blockingcollection instead.

while (true) {
   var nextPacket = _packets.Take();
   ...
}

If your packages are independent (which they should be since you use UDP) you can fire up several worker threads.