Question

When determining whether or not a file exists, how does using the try statement avoid a “race condition”?

I’m asking because a highly upvoted answer (update: it was deleted) seems to imply that using os.path.exists() creates an opportunity that would not exist otherwise.

The example given is:

try:
   with open(filename): pass
except IOError:
   print 'Oh dear.'

But I’m not understanding how that avoids a race condition compared to:

if not os.path.exists(filename):
    print 'Oh dear.'

How does calling os.path.exists(filename) allow the attacker to do something with the file that they could not already do?

Answer 1

The race condition is, of course, between your program and some other code that operates on file (race condition always requires at least two parallel processes or threads, see this for details). That means using open() instead of exists() may really help only in two situations:

1. You check for existence of a file that is created or deleted by some background process (however, if you run inside a web server, that often means there are many copies of your process running in parallel to process HTTP requests, so for web apps race condition is possible even if there are no other programs).
2. There may be some malicious program running that is trying to crash your code by destroying the file at the moments you expect it to exist.

exists() just performs a single check. If file exists, it may be deleted a microsecond after exists() returned True. If file is absent, it may be created immediately.

However, open() not just tests for file existence, but also opens the file (and does these two actions atomically, so nothing can happen between the check and the opening). Usually files can not be deleted while they are open by someone. That means that inside with you may be completely sure: file really exists now since it is open. Though it’s true only inside with, and the file still may be deleted immediately after with block exits, putting code that needs file to exist inside with guarantees that code will not fail.