Question

When using setuptools/distutils to build C libraries in Python

$ python setup.py build

the *.so/*.pyd files are placed in build/lib.win32-2.7 (or equivalent).

I’d like to test these files in my test suite, but I’d rather not hard code the build/lib* path. Does anyone know how to pull this path from distutils so I can sys.path.append(build_path) – or is there an even better way to get hold of these files? (without having installed them first)

Answer 1

You must get the platform that you are running on and the version of python you are running on and then assemble the name yourself. This is an internal detail of setuptools (based on the bundled version of distutils), and so is not guaranteed to be stable.

To get the current platform, use sysconfig.get_platform(). To get the python version, use sys.version_info (specifically the first three elements of the returned tuple). On my system (64-bit linux with python 2.7.2) I get:

>>> import sysconfig
>>> import sys
>>> sysconfig.get_platform()
linux-x86_64
>>> sys.version_info[:3]
(2, 7, 2)

Before setuptools 62.1

The format of the lib directory is "lib.platform-versionmajor.versionminor" (i.e. only 2.7, not 2.7.2). You can construct this string using python’s string formatting methods:

def distutils_dir_name(dname):
    """Returns the name of a distutils build directory"""
    f = "{dirname}.{platform}-{version[0]}.{version[1]}"
    return f.format(dirname=dname,
                    platform=sysconfig.get_platform(),
                    version=sys.version_info)

After setuptools 62.1

The directory above clashes with alternative python implementations like PyPy, so the internal build directory was made more specific using sys.implementation.cache_tag. The format of the lib directory is "lib.<platform>-<cachetag>". You can construct this string using python’s string formatting methods:

def distutils_dir_name(dname):
    """Returns the name of a distutils build directory"""
    f = "{dirname}.{platform}-{cache_tag}"
    return f.format(dirname=dname,
                    platform=sysconfig.get_platform(),
                    cache_tag=sys.implementation.cache_tag)

You can use this to generate the name of any of distutils build directory:

>>> import os
# Before Setuptools 62.1
>>> os.path.join('build', distutils_dir_name('lib'))
build/lib.linux-x86_64-3.9
# After Setuptools 62.1
>>> os.path.join('build', distutils_dir_name('lib'))
build/lib.linux-x86_64-cpython39

Note that Setuptools 62.1 is only available on Python 3.7+. Also, if you set SETUPTOOLS_USE_DISTUTILS=stdlib in your environment, then you will get the old behavior even in Setuptools 62.1.