Question

def get_pkgs():
    pkgs = []
    for importer, modname, ispkg in \
            pkgutil.walk_packages(path=None, onerror=lambda x: None):
    pkgs.append(modname)
    return pkgs

Above code snippet gives me all python packages in distribution. Can someone suggest me a way to get all packages out of these used by a python application ?

Answer 1

Snakefood is nice, as already answered by Sven Marnach, but this gives all file dependencies and I’m not sure if it will just tell you what packages are used, versus every single file dependency.

For packages available in the Python Package Index, you could use virtualenv (and pip, which comes with it) to get a simple list of required/used packages.

For example, assuming you also have the excellent virtualenvwrapper tools installed (highly recommended), below is a sequence that shows what the package requirements are for pylint:

$ mkvirtualenv pylint_dep_check --no-site-packages
New python executable in pylint_dep_check/bin/python
Installing setuptools............done.
$ pip freeze   #Note the wsgiref 'bug' where it always shows up
wsgiref==0.1.2
$ workon pylint_dep_check
(pylint_dep_check) $ pip install pylint
(... snipped lengthy install text ...)
(pylint_dep_check) $ pip freeze
logilab-astng==0.21.1
logilab-common==0.55.0
pylint==0.23.0
unittest2==0.5.1
wsgiref==0.1.2

Noe the all-important use of the --no-site-packages virtualenv creation option which (surprise!) ensures that your virtualenv is completely fresh and has none of the site-packages from your distribution installed. This way it is clear what is needed for the app you installed.

If this is an application that you have developed, a nice way to keep track of your dependencies (and an excellent/clean way to work) is to set up the application in a clean virtualenv (created using the --no-site-packages option again) and then again use pip freeze to figure out what packages you’ve installed to make it work.

The ability to start a “fresh” python installation with the --no-site-packages option is extremely useful. I do it for all applications and to test out packages I’m interested in without cluttering my workspace(s).

If you aren’t using virtualenv and pip yet, get on it already. Here is a good introduction:
http://mathematism.com/2009/07/30/presentation-pip-and-virtualenv/