Question

I’m using the foo module which includes a few module level variables that dictate its behavior. One of these variables is SANITIZE.

I want to be able to use foo in two ways, foo.parse() and foo2.parse(), where the only difference is foo has SANITIZE=false and foo2 has SANITIZE=true

I want to do this while not having to copy paste my code. For example

#foo module
SANITIZE='foo'
def parse():
    print SANITIZE

#foo2 module
import foo
foo.SANITIZE='foo2'


#test script
import foo
import foo2
foo2.foo.parse() #i want this to print 'foo2'
foo.parse() #i want this to print 'foo'

However, the above example will print ‘foo2’ both times. Is there a way to achieve this behavior?

Thanks!

Answer 1

If this is your code, than the solution is not to depend in module level variables, but in some other way to keep state of the objects. Modules in Python are “singletons” – which means that once imported by any module, there is just one version of then, interpreter wide –
The behavior you want, for example, is exactly what you get, for free, if you use class inheritance – the child class can customize some, but no need to rewrite all, of the parent class.

So, if you as much as encapsulate your “foo” code inside a class – you may not even have to write a class that needs instances, you get the features you want already:

#foo module:

class Foo(object):
    sanitize = "value 1"
    @classmethod
    def parse(cls):
        print cls.sanitize

#foo2
from foo import Foo
class Foo2(Foo):
    sanitize = "value 2"

# code:
from foo import Foo
from foo2 import Foo2

Foo.parse()
Foo2.parse()

Of course, with the cheer amount of introspection and metaprogramming that Python allows, it would be possible to do something like what you want – but it would complicate matters, for no good. Module variables have most of the same disadvantages that global variables have in C code.

One way to do it is to make “foo” when accessed through foo2 to make the variable changes on the fly, call the function in “foo” and restore the previosu values on exit.
For arbitrary code to be triggered on attribute access of “foo” on the module “foo2”, “foo” has to refer to an object with a “property” attribute —

So, the exa example you wrote would run, in a concurency-unsafe way, btw, very unsafe way, if foo2 is written more or less so:

import foo as _foo

SANITIZE = "value2"

class _FooWrapper(object):
    def __getattribute__(self, attr):
        self.change_vars()
        original_function = getattr(_foo, attr)
        if callable(original):
            def wrapper(func):
                def new_func(*args, **kw):
                    res = func(*args, **kw)
                    self.restore_vars()
                    return res
                return new_func
            return wrapper(original)
        return original

    def change_vars(self):
        self.original_sanitize = _foo.SANITIZE
        _foo.SANITIZE = SANITIZE
    def restore_vars(self):
        __foo.SANITIZE = self.original_sanitize

foo = _FooWrapper()

This creates a “foo” object in module foo2 that when accessed, retrieves any requested attributes from the original “foo” module instead. So “foo2.foo.parse” will get the “foo” parse function – but, perceive the ammount of “hackiness” in this approach – in order to be able to restore the original value in the “foo” module that lives inside the Python interpreter, after the function was fetched from foo2, that function have too, upon returning, restore the values itself. The only way to do so is modifying the function so that it runs additional code when it returns – so it is decorated on the fly by the code above.

I think this example makes clear that having module level configurations is not the way to go in this case, although possible.

EDIT
The O.P. commented:

Thanks jsbueno, unfortunately this is not my code and I must rely on
the module level variables. The Wrapper class method is interesting
but as you said, very hacky and incredibly non thread safe, which I’m
afraid won’t do for my case

In reply to that:

Modules are “singletons” – so, changing the variable on the module, at
any point will make it thread unsafe. The other way I can think about
this is creating a “photocopier” module that actually re-creates
classes, attributes and instances of another, existing module, when
imported – rebinding all functions global variables (methods would
still be acessed as functions at this level)

Reading this description it may sound as “unfeasable” – but it is easier done than described –
Here follows my “foo2” module that does the above:

from types import ModuleType, FunctionType

import foo as _foo

SANITIZE = "value 2"

def rebuild_function(func, glob):
    """Rebinds the globals in the given functions to globals in 
    this module  by default"""
    new_func = FunctionType(func.func_code,
                            glob,
                            func.func_name,
                            func.func_defaults,
                            func.func_closure)
    return new_func

def rebuild_class(cls, glob):
    metatype = type(cls)
    dct = cls.__dict__.copy()
    for key, value in dct.items():
        if isinstance(value, FunctionType):
            dct[key] = rebuild_function(value, glob)
    return metatype(cls.__name__, cls.__bases__, dct)

def rebuild_module(mod,glob):
    new_module = ModuleType(mod.__name__)
    for key, value in mod.__dict__.items():
        if isinstance(value, FunctionType):
            value = rebuild_function(value, glob)
        elif isinstance(value, type):
            value = rebuild_class(value, glob)
        setattr(new_module, key, value)
    return new_module

foo = rebuild_module(_foo, globals())

__all__ = ["foo", "SANITIZE"]

This code does exactly what I described – it recreates all function objects in the original module, rebindign the globals dict for each function. It is concurrency safe. There are some corner cases if the to-be clonned module does point to native code classes or functions (they are not of “FunctionType”). If it makes heavy usage of multiple class inheritance, metaclasses,- it miught work, or not.

I tested it with a simple class and it worked fine:

#module "foo"
SANITIZE='foo'
def parse():
    print SANITIZE

class Parser(object):
    def __init__(self):
        print SANITIZE * 2

And

#test script
import foo
import foo2
foo2.foo.parse() #i want this to print 'foo2'
foo2.foo.Parser()
foo.parse() #i want this to print 'foo'
foo.Parser()

Output:

[gwidion@powerpuff tmp16]$ python test_foo.py 
foofoo
value 2
value 2value 2
foo
foofoo
[gwidion@powerpuff tmp16]$