Question

I am working on an user space app for an embedded Linux project using the 2.6.24.3 kernel.
My app passes data between two file nodes by creating 2 pthreads that each sleep until a asynchronous IO operation completes at which point it wakes and runs a completion handler.

The completion handlers need to keep track of how many transfers are pending and maintain a handful of linked lists that one thread will add to and the other will remove.

// sleep here until events arrive or time out expires
for(;;) {
    no_of_events = io_getevents(ctx, 1, num_events, events, &timeout);
    // Process each aio event that has completed or thrown an error
    for (i=0; i<no_of_events; i++) {
        // Get pointer to completion handler
        io_complete = (io_callback_t) events[i].data;
        // Get pointer to data object
        iocb = (struct iocb *) events[i].obj;
        // Call completion handler and pass it the data object
        io_complete(ctx, iocb, events[i].res, events[i].res2);
    }
}

My question is this…

Is there a simple way I can prevent the currently active thread from yielding whilst it runs the completion handler rather than going down the mutex/spin lock route?

Or failing that can Linux be configured to prevent yielding a pthread when a mutex/spin lock is held?

Answer 1

You can use the sched_setscheduler() system call to temporarily set the thread’s scheduling policy to SCHED_FIFO, then set it back again. From the sched_setscheduler() man page:

A SCHED_FIFO process runs until either
it is blocked by an I/O request, it is
preempted by a higher priority
process, or it calls sched_yield(2).

(In this context, “process” actually means “thread”).

However, this is quite a suspicious requirement. What is the problem you are hoping to solve? If you are just trying to protect your linked list of completion handlers from concurrent access, then an ordinary mutex is the way to go. Have the completion thread lock the mutex, remove the list item, unlock the mutex, then call the completion handler.