Question

Is it possible to programmatically detect dependencies given a python project residing in SVN?

Answer 1

Here is a twist which adds some precision, and which might be useful if you find you’re frequently checking dependencies of miscellaneous code:

• Catches only import statements executed by the code being analyzed.
• Automatically excludes all system-loaded modules, so you don’t have to weed through it.
• Also reports the symbols imported from each module.

Code:

import __builtin__
import collections
import sys

IN_USE = collections.defaultdict(set)
_IMPORT = __builtin__.__import__

def _myimport(name, globs=None, locs=None, fromlist=None, level=-1):
    global IN_USE
    if fromlist is None:
        fromlist = []
    IN_USE[name].update(fromlist)
    return _IMPORT(name, globs, locs, fromlist, level)

# monkey-patch __import__
setattr(__builtin__, '__import__', _myimport)

# import and run the target project here and run the routine
import foobar
foobar.do_something()

# when it finishes running, dump the imports
print 'modules and symbols imported by "foobar":'
for key in sorted(IN_USE.keys()):
    print key
    for name in sorted(IN_USE[key]):
        print '  ', name

Example foobar module:

import byteplay
import cjson

def _other():
    from os import path
    from sys import modules

def do_something():
    import hashlib
    import lxml
    _other()

Output:

modules and symbols imported by "foobar":
_hashlib
array
   array
byteplay
cStringIO
   StringIO
cjson
dis
   findlabels
foobar
hashlib
itertools
lxml
opcode
   *
   __all__
operator
os
   path
sys
   modules
types
warnings