Question

I would like to be able to force a context switch from one thread to another. Therefore, I have implemented the following locking procedure:

#define TRUE  (1==1)
#define FALSE (0==1)

#include <pthread.h>

int acquire(void);
int release(void);
int c_yield(int count);

// Who was the last to acquire the lock
static volatile pthread_t lock_owner;

// Is the lock currently taken
static volatile int lock_taken = FALSE;

/* This variable indicates how many threads are currently waiting for
 * the lock. */
static volatile int lock_wanted = 0;

/* Mutex for protecting access to lock_wanted, lock_owner and
 * lock_taken */
static pthread_mutex_t mutex;

/* Condition even to notify when the lock becomes available */
static pthread_cond_t cond;

void init_lock(void) {
    pthread_cond_init(&cond, NULL);
    pthread_mutex_init(&mutex, NULL);
}

int acquire(void) {
    pthread_mutex_lock(&mutex);
    if(lock_taken) {
        lock_wanted++;
        pthread_cond_wait(&cond, &mutex);
        lock_wanted--;
    } 
    if(lock_taken) {
        pthread_mutex_unlock(&mutex);
        return EPROTO;
    } 
    lock_taken = TRUE;
    lock_owner = pthread_self();
    return pthread_mutex_unlock(&mutex);
}

int release(void) {
    pthread_mutex_lock(&mutex);
    lock_taken = FALSE;
    if(lock_wanted > 0) {
        pthread_cond_signal(&cond);
    }
    return pthread_mutex_unlock(&mutex);
}

Using another method (not shown), I can then implement a yield() that only returns if there are either no threads waiting for the lock, or after at least one other thread had a chance to run.

This implementation works fine most of the time, but if I stress-test it with ~50 threads trying to acquire and release the lock in random intervals, every once in a while acquire() will return EPROTO, indicating that someone called pthread_cond_signal without setting first lock_taken = FALSE.

Why is that? It seems as if the CPU sometimes doesn’t see the new value of lock_taken, which is why I already made the variables volatile. But it’s still happening…

Answer 1

if(lock_taken) {
    lock_wanted++;
    pthread_cond_wait(&cond, &mutex);
    lock_wanted--;
} 

This should be while(lock_taken), not if. There are several reasons you might wake from pthread_cond_wait but find the lock taken by another thread by the time you get scheduled. One is if there’s a spurious wakeup. The other is if another thread enters acquire after we block, finds the lock not taken, and takes it itself before this thread gets the mutex again.

The canonical way would be:

lock_wanted++;
while(lock_taken) pthread_cond_wait(&cond, &mutex);
lock_wanted--;

Get rid of all the volatiles, they harm performance and are not needed. Because mutexes are sufficient synchronization, nothing else is necessary. (And, to anyone else looking at your code, they signal that you don’t understand thread synchronization and tried to ‘sprinkle them in’ until it just happened to work.)