Question

I am writing a simple scheduling service. I don’t want to hard-code all of the tasks it can schedule and instead would like to support plugins that can be dropped in a folder and loaded dynamically at runtime.

My plan is to have a JSON file (or any configuration file) that maps a task name to the location of a Python file (a module) which will have a class called Plugin. Pretty simple I thought. When someone schedules a task to run, they pass the task name and the time to run it. When the time elapses, the plugin is loaded (or reloaded) and is ran with any additional arguments passed to the scheduler.

I have been looking at the imp module to see how to load modules at runtime. I am not sure whether I want to list plugins using their physical location (file system path) or to use their module names like you’d see in a import statement. It seems imp wants to use physical location.

I got two different versions of this code “working”. Here is one that uses importlib:

    pluginName = self.__pluginLookup[pluginName]
    module = import_module(pluginName)
    module = reload(module)  # force reload

    plugin = module.Plugin()
    return plugin

This is one I wrote using imp:

    path = self.__pluginLookup[pluginName]
    path, moduleName = split(path)
    moduleName, extension = splitext(moduleName)

    file, path, description = find_module(moduleName, [path])
    with file:
        module = load_module(moduleName, file, path, description)

    plugin = module.Plugin()
    return plugin

The problem I am running into is handling dependencies. If I have a plugin.py file that depends on a dependency.py file in the same folder, saying import dependency doesn’t seem to work. Instead, it looks up the dependency from the PYTHONPATH.

How can I make the imports relative to the plugins themselves?

Answer 1

You could append path to sys.path:

import sys
sys.path.append(path)

where path is the directory containing the dependency.py.