Question

Are Python docstrings and comments stored in memory when a module is loaded?

I’ve wondered if this is true, because I usually document my code well; may this affect memory usage?

Usually every Python object has a __doc__ method. Are those docstrings read from the file, or processed otherwise?

I’ve done searches here in the forums, Google and Mailing-Lists, but I haven’t found any relevant information.

Do you know better?

Answer 1

By default, docstrings are present in the .pyc bytecode file, and are loaded from them (comments are not). If you use python -OO (the -OO flag stands for “optimize intensely”, as opposed to -O which stands for “optimize mildly), you get and use .pyo files instead of .pyc files, and those are optimized by omitting the docstrings (in addition to the optimizations done by -O, which remove assert statements). E.g., consider a file foo.py that has:

"""This is the documentation for my module foo."""

def bar(x):
  """This is the documentation for my function foo.bar."""
  return x + 1

you could have the following shell session…:

$ python -c'import foo; print foo.bar(22); print foo.__doc__'
23
This is the documentation for my module foo.
$ ls -l foo.pyc
-rw-r--r--  1 aleax  eng  327 Dec 30 16:17 foo.pyc
$ python -O -c'import foo; print foo.bar(22); print foo.__doc__'
23
This is the documentation for my module foo.
$ ls -l foo.pyo
-rw-r--r--  1 aleax  eng  327 Dec 30 16:17 foo.pyo
$ python -OO -c'import foo; print foo.bar(22); print foo.__doc__'
23
This is the documentation for my module foo.
$ ls -l foo.pyo
-rw-r--r--  1 aleax  eng  327 Dec 30 16:17 foo.pyo
$ rm foo.pyo
$ python -OO -c'import foo; print foo.bar(22); print foo.__doc__'
23
None
$ ls -l foo.pyo
-rw-r--r--  1 aleax  eng  204 Dec 30 16:17 foo.pyo

Note that, since we used -O first, the .pyo file was 327 bytes — even after using -OO, because the .pyo file was still around and Python didn’t rebuild/overwrite it, it just used the existing one. Removing the existing .pyo (or, equivalently, touch foo.py so that Python knows the .pyo is “out of date”) means that Python rebuilds it (and, in this case, saves 123 bytes on disk, and a little bit more when the module’s imported — but all .__doc__ entries disappear and are replaced by None).