Question

I’m writing a modular application that will dynamically import modules at runtime based upon launch parameters, etc. To keep things organised, I have the following directory structure:

    myapp/
    ├── __init__.py
    ├── main.py
    ├── utils.py
    ├──modules/
        ├── __init__.py
        ├── iptables/
        │   ├── __init__.py
        │   ├── iptables.py
        │   ├── [other files/directories/configuration...]
        │   └── tests
        │       ├── iptables_tests.py
        ├── layer4/
        │   ├── __init__.py
        │   ├── layer4.py
        │   ├── [other files/directories/configuration...]
        │   └── tests
        │       ├── layer4_tests.py

I’m trying to accomplish the following:

• main.py should be able to do something like from modules.layer4 import Layer4Manager, with Layer4Manager being a class inside modules/layer4/layer4.py. I’d like to do this without including the actual code inside __init__.py if possible, as there will be other scripts/files in the same directory, so it makes sense for them to be grouped together in directories (this is also why the scripts aren’t just in the modules directory).

• iptables.py and layer4.py should be able to import functions from utils.py.

• Tests using the Python unittest framework and Nose should run correctly by, for example, iptables_tests.py should be able to runfrom myapp.modules.iptables import IPTablesController and from myapp.utils import UtilityFunction.

I’ve been battling with this for a few hours now and I’m unable to get all three of the above items working at the same time. Using from layer4 import * or the __all__ variable inside modules/layer4/__init__.py breaks the tests, as they are unable to from myapp.utils import UtilityFunction.

Blanking modules/*/__init__.py stops me from being able to from myapp.modules.layer4 import Layer4Manager.

I’ve tried several other combinations and I think I’m just making matters worse. I’m targeting Python 2.6. I’m guessing some kind of relative import will make things easier, but I’m not sure how they would work with the two-level directory structure I’m using here.

Thanks in advance!

Answer 1

You don’t need main/__init__.py because you don’t want to make myapp a package and when you will execute main.py, myapp will be automatically added to python path.

Other than that it looks fine, you should be able to directly import modules from subpackages if you put the correct paths e.g.

import utils
from modules.layer4 import layer4

I created a similar structure and it works well e.g.

$ cat myapp/main.py 
import utils
from modules.layer4 import layer4
from modules.iptables import iptables

$ cat myapp/modules/iptables/iptables.py
from modules.layer4 import layer4

$ cat myapp/modules/layer4/layer4.py
import utils

and all this works without error