Question

How can a Python program easily import a Python module from a file with an arbitrary name?

The standard library import mechanism doesn’t seem to help. An important constraint is that I don’t want an accompanying bytecode file to appear; if I use imp.load_module on a source file named foo, a file named fooc appears, which is messy and confusing.

The Python import mechanism expects that it knows best what the filename will be: module files are found at specific filesystem locations, and in particular that the filenames have particular suffixes (foo.py for Python source code, etc.) and no others.

That clashes with another convention, at least on Unix: files which will be executed as a command should be named without reference to the implementation language. The command to do “foo”, for example, should be in a program file named foo with no suffix.

Unit-testing such a program file, though, requires importing that file. I need the objects from the program file as a Python module object, ready for manipulation in the unit test cases, exactly as import would give me.

What is the Pythonic way to import a module, especially from a file whose name doesn’t end with .py, without a bytecode file appearing for that import?

Answer 1

My best implementation so far is (using features only in Python 2.6 or later):

import os
import sys
import imp
import contextlib

@contextlib.contextmanager
def preserve_value(namespace, name):
    """ A context manager to preserve, then restore, the specified binding.

        :param namespace: The namespace object (e.g. a class or dict)
            containing the name binding.
        :param name: The name of the binding to be preserved.
        :yield: None.

        When the context manager is entered, the current value bound to
        `name` in `namespace` is saved. When the context manager is
        exited, the binding is re-established to the saved value.

        """
    saved_value = getattr(namespace, name)
    yield
    setattr(namespace, name, saved_value)


def make_module_from_file(module_name, module_filepath):
    """ Make a new module object from the source code in specified file.

        :param module_name: The name of the resulting module object.
        :param module_filepath: The filesystem path to open for
            reading the module's Python source.
        :return: The module object.

        The Python import mechanism is not used. No cached bytecode
        file is created, and no entry is placed in `sys.modules`.

        """
    py_source_open_mode = 'U'
    py_source_description = (".py", py_source_open_mode, imp.PY_SOURCE)

    with open(module_filepath, py_source_open_mode) as module_file:
        with preserve_value(sys, 'dont_write_bytecode'):
            sys.dont_write_bytecode = True
            module = imp.load_module(
                    module_name, module_file, module_filepath,
                    py_source_description)

    return module


def import_program_as_module(program_filepath):
    """ Import module from program file `program_filepath`.

        :param program_filepath: The full filesystem path to the program.
            This name will be used for both the source file to read, and
            the resulting module name.
        :return: The module object.

        A program file has an arbitrary name; it is not suitable to
        create a corresponding bytecode file alongside. So the creation
        of bytecode is suppressed during the import.

        The module object will also be added to `sys.modules`.

        """
    module_name = os.path.basename(program_filepath)

    module = make_module_from_file(module_name, program_filename)
    sys.modules[module_name] = module

    return module

This is a little too broad: it disables bytecode file generation during the entire import process for the module, which means other modules imported during that process will also not have bytecode files generated.

I’m still looking for a way to disable bytecode file generation for only the specified module file.