Question

I wish to have two threads. The first thread1 occasionally calls the following pseudo function:

void waitForThread2() {
  if (thread2 is not idle) {
    return;
  }
  notifyThread2IamReady(); // i.e. via 1st condition variable
  Wait for thread2 to finish exclusive access. // i.e. via 2nd condition variable.
}

The second thread2 is forever in the following pseudo loop:

for (;;) {
  Notify thread1 I am idle.
  Wait for thread1 to be ready. // i.e. via 1st condition variable.
  Notify thread1 I am exclusive.
  Do some work while thread1 is blocked.
  Notify thread1 I am busy. // i.e. via 2nd condition variable.
  Do some work in parallel with thread1.
}

What is the best way to write this such that both thread1 and thread2 are kept as busy as possible on a machine with multiple cores. I would like to avoid long delays between notification in one thread and detection by the other. I tried using pthread condition variables but found the delay between thread2 doing ‘notify thread1 I am busy’ and the loop in waitForThread2() on thear2IsExclusive() can be up to almost one second delay. I then tried using a volatile sig_atomic_t shared variable to control the same, but something is going wrong, so I must not be doing it correctly.

Answer 1

I looks to me like you are trying to do a rendezvous (a term from Ada).

The second thread is sitting, waiting for the first thread to call it, then it does some work immediately while the first thread waits, and some more work after the first thread is finished.

The first thread is “calling” the second thread – with an immediate timeout if the second thread is unavailable to take the call.

Ada supports this directly in the language, but assuming that porting to Ada isn’t an option…

This could be implemented with three semaphores. Semaphore 1 indicates that thread 1 is ready to rendezvous. Semaphore 2 indicates thread 2 is ready to rendevous. Semaphore 3 indicates the rendezvous is complete.

Thread 1: Defaults with Semaphore 1 acquired.

 if Semaphore 2.acquire(timeout = 0) is successful # Thread 2 is ready
     Semaphore 1.release() # Indicate I am ready
     Semaphore 3.acquire() # Wait until the rendevous is complete.
     Semaphore 3.release()
     Semaphore 1.acquire() # Indicate I am not ready
     Semaphore 2.release() # I am no longer using thread 2.

 Do concurrent work 

Thread 2: Defaults with Semaphore 2 acquired.

 Loop forever
     Semaphore 3.acquire() # Indicate Rendevous is not complete.
     Semaphore_2.release() # Indicate I am ready
     Semaphore_1.acquire() # Wait for Thread 1 to be ready
     Joint processing
     Semaphore 1.release() # I am no longer using thread 1.
     Semaphore 3.release() # Rendevous is complete.
     Semaphore 2.acquire() # I am not ready

 Post-processing

NOTE: Written from scratch, not tested. Looks a lot more complicated than I thought it would be when I started; have I missed something?