Question

I’m developing a python framework that would have ‘addons’ written as separate packages. I.e.:

import myframework from myframework.addons import foo, bar 

Now, what I’m trying to arrange is so that these addons can be distributed separately from core framework and injected into myframework.addons namespace.

Currently my best solution to this is the following. An add-on would be deployed (most likely into {python_version}/site-packages/ like so:

fooext/ fooext/__init__.py fooext/myframework/ fooext/myframework/__init__.py fooext/myframework/addons/ fooext/myframework/addons/__init__.py fooext/myframework/addons/foo.py 

The fooext/myframework/addons/__init__.py would have the pkgutil path extension code:

import pkgutil __path__ = pkgutil.extend_path(__path__, __name__) 

The problem is that for this to work, the PYTHONPATH needs to have fooext/ in it, however the only thing it would have is the parent install directory (most likely, the above-mentioned site-packages).

The solution to this is to have extra code in myframework/addons/__init__.py which would tranverse sys.path and look for any modules with a myframework sub-package, in which case it adds it to sys.path and everything works.

Another idea I had is to write the addon files directly to myframework/addons/ install location, but then it would make development and deployed namespace differ.

Is there a better way to accomplish this or perhaps a different approach to the above distribution problem altogether?

Answer 1

Is there a better way to accomplish this or perhaps a different approach to the above distribution problem altogether?

Possibly. Python’s module/package setup is generally tricky to tamper with dynamically like this, but its object/class system is open and extensible in a well-defined way. When modules and packages don’t quite have the features you need to encapsulate your project nicely you can use classes instead.

For example you could have the extension functionality in a completely different package, but allow it to inject classes into your basic framework through a specific interface. eg. myframework/_​​_​init​_​​_.py containing a basic application wrapper:

class MyFramework(object):     '''A bare MyFramework, I only hold a person's name     '''     _addons= {}     @staticmethod     def addAddon(name, addon):         MyFramework._addons[name]= addon      def __init__(self, person):         self.person= person         for name, addon in MyFramework._addons.items():             setattr(self, name, addon(self)) 

Then you could have extension functionality in a myexts/helloer.py, that keeps a reference to its ‘owner’ or ‘outer’ MyFramework class instance:

class Helloer(object):     def __init__(self, owner):         self.owner= owner     def hello(self):         print 'hello '+self.owner.person  import myframework myframework.MyFramework.addAddon('helloer', Helloer) 

So now if you just ‘import myframework’, you only get the basic functionality. But if you also ‘import myexts.helloer’ you also get the ability to call MyFramework.helloer.hello(). Naturally you can also define protocols for addons to interact with the basic framework behaviour, and each other. You can also do things like inner classes a subclass of the framework can override to customise without having to monkey-patch classes that might affect other applications, if you need that level of complexity.

Encapsulating behaviour like this can be useful, but it’s typically annoying work to adapt module-level code you’ve already got to fit this model.