Question

Is there any easy way to have a system-wide mutex in Python on Linux? By “system-wide”, I mean the mutex will be used by a group of Python processes; this is in contrast to a traditional mutex, which is used by a group of threads within the same process.

EDIT: I’m not sure Python’s multiprocessing package is what I need. For example, I can execute the following in two different interpreters:

from multiprocessing import Lock
L = Lock()
L.acquire()

When I execute these commands simultaneously in two separate interpreters, I want one of them to hang. Instead, neither hangs; it appears they aren’t acquiring the same mutex.

Answer 1

The “traditional” Unix answer is to use file locks. You can use lockf(3) to lock sections of a file so that other processes can’t edit it; a very common abuse is to use this as a mutex between processes. The python equivalent is fcntl.lockf.

Traditionally you write the PID of the locking process into the lock file, so that deadlocks due to processes dying while holding the lock are identifiable and fixable.

This gets you what you want, since your lock is in a global namespace (the filesystem) and accessible to all processes. This approach also has the perk that non-Python programs can participate in your locking. The downside is that you need a place for this lock file to live; also, some filesystems don’t actually lock correctly, so there’s a risk that it will silently fail to achieve exclusion. You win some, you lose some.