Question

I have a multi-threaded application that is using pthreads. I have a mutex() lock and condition variables(). There are two threads, one thread is producing data for the second thread, a worker, which is trying to process the produced data in a real time fashion such that one chuck is processed as close to the elapsing of a fixed time period as possible.

This works pretty well, however, occasionally when the producer thread releases the condition upon which the worker is waiting, a delay of up to almost a whole second is seen before the worker thread gets control and executes again.

I know this because right before the producer releases the condition upon which the worker is waiting, it does a chuck of processing for the worker if it is time to process another chuck, then immediately upon receiving the condition in the worker thread, it also does a chuck of processing if it is time to process another chuck.

In this later case, I am seeing that I am late processing the chuck many times. I’d like to eliminate this lost efficiency and do what I can to keep the chucks ticking away as close to possible to the desired frequency.

Is there anything I can do to reduce the delay between the release condition from the producer and the detection that that condition is released such that the worker resumes processing? For example, would it help for the producer to call something to force itself to be context switched out?

Bottom line is the worker has to wait each time it asks the producer to create work for itself so that the producer can muck with the worker’s data structures before telling the worker it is ready to run in parallel again. This period of exclusive access by the producer is meant to be short, but during this period, I am also checking for real-time work to be done by the producer on behalf of the worker while the producer has exclusive access. Somehow my hand off back to running in parallel again results in significant delay occasionally that I would like to avoid. Please suggest how this might be best accomplished.

Answer 1

I could suggest the following pattern. Generally the same technique could be used, e.g. when prebuffering frames in some real-time renderers or something like that.

First, it’s obvious that approach that you describe in your message would only be effective if both of your threads are loaded equally (or almost equally) all the time. If not, multi-threading would actually benefit in your situation.

Now, let’s think about a thread pattern that would be optimal for your problem. Assume we have a yielding and a processing thread. First of them prepares chunks of data to process, the second makes processing and stores the processing result somewhere (not actually important).

The effective way to make these threads work together is the proper yielding mechanism. Your yielding thread should simply add data to some shared buffer and shouldn’t actually care about what would happen with that data. And, well, your buffer could be implemented as a simple FIFO queue. This means that your yielding thread should prepare data to process and make a PUSH call to your queue:

X = PREPARE_DATA()
BUFFER.LOCK()
BUFFER.PUSH(X)
BUFFER.UNLOCK()

Now, the processing thread. It’s behaviour should be described this way (you should probably add some artificial delay like SLEEP(X) between calls to EMPTY)

IF !EMPTY(BUFFER) PROCESS(BUFFER.TOP)

The important moment here is what should your processing thread do with processed data. The obvious approach means making a POP call after the data is processed, but you will probably want to come with some better idea. Anyway, in my variant this would look like

// After data is processed
BUFFER.LOCK()
BUFFER.POP()
BUFFER.UNLOCK()

Note that locking operations in yielding and processing threads shouldn’t actually impact your performance because they are only called once per chunk of data.

---

Now, the interesting part. As I wrote at the beginning, this approach would only be effective if threads act somewhat the same in terms of CPU / Resource usage. There is a way to make these threading solution effective even if this condition is not constantly true and matters on some other runtime conditions.

This way means creating another thread that is called controller thread. This thread would merely compare the time that each thread uses to process one chunk of data and balance the thread priorities accordingly. Actually, we don’t have to “compare the time”, the controller thread could simply work the way like:

IF BUFFER.SIZE() > T
   DECREASE_PRIORITY(YIELDING_THREAD)
   INCREASE_PRIORITY(PROCESSING_THREAD)

Of course, you could implement some better heuristics here but the approach with controller thread should be clear.