Question

Given the following example layout:

test/
  test.py
  formats/
    __init__.py
    format_a.py
    format_b.py

What I try to archive is, that whenever I import formats, the __init__.py looks for all available modules in the formats subdir, loads them and makes them available (right now simply through a variable, supported_formats). If theres a better, more pythonic or otherwise approach to dynamically loading stuff on runtime, based on physical available files, please tell.

My Approach

I tried something like this (in __init__.py):

supported_formats =  [__import__(f[:f.index('.py')]) for f in glob.glob('*.py')]

So far I just get it to work, when I run __init__.py from the command line (from the formats subdir or from other directories) . But when I import it from test.py, it bails on me like this:

ImportError: No module named format_a.py

Same when I import it from the python interpreter, when I started the interpreter in an other directory but the formats subdir.

Here’s the whole code. It also looks for a specific class and stores one instance of each class in an dict, but loading the modules dynamically is the main part I don’t get:

def dload(get_cls=True, get_mod=True, key=None, fstring_mod='*.py', fstring_class=''):
  if p.dirname(__file__):
    path = p.split(p.abspath(__file__))[0]
    fstring_mod = p.join(path, fstring_mod)
    print >> sys.stderr, 'Path-Glob:', fstring_mod
  modules = [p.split(fn)[1][:fn.index('.py')] for fn in glob.glob(fstring_mod)]
  print >> sys.stderr, 'Modules:', ', '.join(modules)
  modules = [__import__(m) for m in modules]
  if get_cls:
    classes = {} if key else []
    for m in modules:
      print >> sys.stderr, "-", m
      for c in [m.__dict__[c]() for c in m.__dict__ if c.startswith(fstring_class)]:
        print >> sys.stderr, " ", c
        if key:
          classes[getattr(c, key)] = c
        else:
          classes.append(c)
    if get_mod:
      return (modules, classes)
    else:
      return classes
  elif get_mod:
    return modules

_supported_formats = dload(get_mod=False, key='fid', fstring_mod='format_*.py', fstring_class='Format')

My Idea

The whole messing with filesystem-paths and the like is probably messy anyway. I would like to handle this with module namespaces or something similar, but I’m kinda lost right now on how start and how to address the modules, so they reachable from anywhere.

Answer 1

There are two fixes you need to make to your code:

1. You should call __import__(m, globals(), locals()) instead of __import__(m). This is needed for Python to locate the modules within the package.

2. Your code doesn’t remove the .py extension properly since you call index() on the wrong string. If it will always be a .py extension, you can simply use p.split(fn)[1][:-3] instead.