Question

In the pickle module documentation there is a snippet of example code:

reader = pickle.load(open('save.p', 'rb'))

which upon first read looked like it would allocate a system file descriptor, read its contents and then “leak” the open descriptor for there isn’t any handle accessible to call close() upon. This got me wondering if there was any hidden magic that takes care of this case.

Diving into the source, I found in Modules/_fileio.c that file descriptors are closed by the fileio_dealloc() destructor which led to the real question.

What is the duration of the file object used by the example code above? After that statement executes does the object indeed become unreferenced and therefore will the fd be subject to a real close(2) call at some future garbage collection sweep? If so, is the example line good practice, or should one not count on the fd being released thus risking kernel per-process descriptor table exhaustion?

Answer 1

What is the duration of the file
object returned by the example code
above?

That code does not return the file object (as the Q’s title correctly says, it receives it as an argument).

In current CPython, the file will be closed at the time the function return (since the function doesn’t stash away any references to the file object to more durable places). In other implementations, the file will be closed “eventually” but there’s no precise time specified.

Depending on CPython’s immediate-closure semantics (which may change when a future CPython moves to better garbage-collection mechanisms), while a very traditional approach, is not best practice.

Rather, best practice is to use the with statement:

with open(...) as f:
  reader = pickle.load(f)

With this use, the file’s immediate closure (as soon as the body of the with statement ends) is guaranteed in all implementations.

Note that, in Python 2.5, you need a from __future__ import with_statement to use with. In 2.6 or better, such an “import from the future” is not required for this purpose (it’s innocuous, but if you know you’re never going to be running under 2.5 it’s redundant and better removed).