Question

I am currently writing a bulk processing algorithm for pitch detection in audio being streamed from disk. I have tightened up my algorithm such that it runs in nearly real-time for serially streamed data.

Ideally I would like the system to be working way faster than real-time such that I could hand it real-time data and after a not huge delay be generating the pitch track data.

Now the thing that strikes me is that the serial processing of the data is where I could provide a hell of a lot of speedup. I’m running on a quad core i7 (with 8 hardware threads) so I ought to be able to improve the speed significantly by spreading processing across multiple blocks.

As it goes I currently do the following:

1. Stream data off disk
2. Buffer data until I have the window size that I wish to analyse.
3. Process the window of data.
4. Copy the data back n-samples (where n is the amount I want to slide (this can be as low as 1ms back in an 80ms window!)
5. rinse and repeat.

Now it strikes me that once I have a window I could easily copy that data into a given thread work buffer (as well as providing a memory location the result will be written to). This way I could, effectively buffer up to 7 (Leave thread 8 open to pump the data) threads worth of data that a thread pool would then process.

When I try to submit the 8th window of audio I want the pool to block until a thread is available to process the data and so on. The idea being that I would keep 7 threads constantly working processing the data. From previous experience I would expect to see about 5x speed up from doing this.

In the past I have written under C++ my own task based system that would do the job perfectly but this app is being developed under C#. To get good parallelism with low overhead under C++ I spent significant amounts of time building a good lockless queueing mechanism.

I was rather hoping, under C#, that someone would have taken the pain out of doing this for me. However I can’t find anything that would seem to work. I’ve looked at System.Threading.ThreadPool and it appears to have no way of checking how many threads are currently in action. Not to mention that the overheads seems prohibitive. The big problem then arises that I can’t re-use an existing pre-allocated structure (which is important in my processing) forcing me to re-create it each time I submit a work item. This has the huge disadvantage that I then generate work quicker than I can process it so not only do I end up wasting tonnes of time setting up structure and workspaces that I really ought not to need but my memory usage spirals out of control.

I then found out about System.Threading.Tasks but that too doesn’t seem to offer the functionality I’m after.

I guess I could just use my C++ Task Manager via interop but I really assumed that in this day and age someone would already have set up something similar. So am I missing something? Or can anyone provide me with a link to such a task management engine?

Answer 1

Task Parallel Library was designed and implemented especially for tasks you are trying to solve! Also you can pipeline this process as well.

So you have to ensure:

• You can run on .NET Framework 4.X
• You are familar with Pipelines design pattern
• You are familar with .NET Framework 4 BlockingCollecition class (BlockingQueue implementation especially to simplify solving of the Producer Consumer problem)