Question

I have a question about how flock() works, particularly in python. I have a module that opens a serial connection (via os.open()). I need to make this thread safe. It’s easy enough making it thread safe when working in the same module using threading.Lock(), but if the module gets imported from different places, it breaks.

I was thinking of using flock(), but I’m having trouble finding enough information about how exactly flock works. I read that flock() unlocks the file once the file is closed. But is there a situation that will keep the file open if python crashes?

And what exactly is allowed to use the locked file if LOCK_EX is set? Just the module that locked the file? Any module that was imported from the script that was originally run?

Answer 1

When a process dies the OS should clean up any open file resources (with some caveats, I’m sure). This is because the advisory lock is released when the file is closed, an operation which occurs as part of the OS cleanup when the python process exits.

Remember, flock(2) is merely advisory:

Advisory locks allow cooperating processes to perform consistent operations on files, but [other, poorly behaved] processes may still access those files without using advisory locks.

flock(2) implements a readers-writer lock. You can’t flock the same file twice with LOCK_EX, but any number of people can flock it with LOCK_SH simultaneously (as long as nobody else has a LOCK_EX on it).

The locking mechanism allows two types of locks: shared locks and exclusive locks. At any time multiple shared locks may be applied to a file, but at no time are multiple exclusive, or both shared and exclusive, locks allowed simultaneously on a file.

flock works at the OS/process level and is independent of python modules. One module may request n locks, or n locks could be requested across m modules. However, only one process can hold a LOCK_EX lock on a given file at a given time.

YMMV on a “non-UNIX” system or a non-local filesystem.