Question

I have one main thread that does some rather CPU intensive operation. The thread has to hold a lock for all its calculations.

Then there are some other threads which occasionally require the same lock for brief amounts of time.

How can I force the main the main thread to occasionally allow the other threads to execute without slowing it down if there are no other threads?

A periodic

lock.release()
time.sleep(x)
lock.acquire()

for some x would pass control to another thread, but slow the main thread down if there are no other threads.

On the other hand, without the sleep() call the GIL seems to interfere here. Since the main thread has the GIL when it executes release(), the other thread is apparently not able to return from acquire() immediately. Therefore, the main thread continues with its own acquire() method and the other threads never get the lock unless a GIL switch happens to coincide exactly with me releasing my own lock.

What’s the proper solution for this problem? Is there a way to force a GIL release? Basically I want some magic piece of code that makes sure that in the following test script the second thread always gets the lock first:

import threading
import time

class t1 (threading.Thread):
    def run(self):
    print "Second thread waiting for lock"
    lock.acquire()
    print "Second thread got lock"
    lock.release()

lock = threading.Lock()
lock.acquire()

t = t1()
t.start()
time.sleep(1)

lock.release()
time.sleep(0) # this works very often, but not all the time
lock.acquire()
print "Main thread got lock"
lock.release()

Answer 1

Merely releasing the GIL doesn’t guarantee that other threads will have a chance to run.

In Unix, the call you really want is sched_yield(). There’s no interface to that function in the Python standard library; it would be straightforward to add one with a native module.

usleep(0) and select() are sometimes used for the same purpose, though it’s not always the same depending on the system scheduler. Windows, you want Sleep(0). Use time.sleep(0) for both of those.