Question

I am about to refactor the code of a python project built on top of twisted. So far I have been using a simple settings.py module to store constants and dictionaries like:

#settings.py
MY_CONSTANT='whatever'
A_SLIGHTLY_COMPLEX_CONF= {'param_a':'a', 'param_b':b}

A great deal of modules import settings.py to do their stuff.

The reason why I want to refactor the project is because I am in need to change/add configuration parameters on the fly. The approach that I am about to take is to gather all configuration in a singleton and to access its instance whenever I need to.

import settings.MyBloatedConfig

def first_insteresting_function():
    cfg = MyBloatedConfig.get_instance()
    a_much_needed_param = cfg["a_respectable_key"]
    #do stuff

#several thousands of functions later

def gazillionth_function_in_module():
    tired_cfg = MyBloatedConfig.get_instance()
    a_frustrated_value = cfg["another_respectable_key"]
    #do other stuff

This approach works but feels unpythonic and bloated. An alternative would be to externalize the cfg object in the module, like this:

CONFIG=MyBloatedConfig.get_instance()

def a_suspiciously_slimmer_function():
    suspicious_value = CONFIG["a_shady_parameter_key"]

Unfortunately this does not work if I am changing the MyBloatedConfig instance entries in another module. Since I am using the reactor pattern, storing staff on a thread local is out of question as well as using a queue.

For completeness, following is the implementation I am using to implement a singleton pattern

instances = {}
def singleton(cls):
    """ Use class as singleton. """
    global instances

    @wraps(cls)
    def get_instance(*args, **kwargs):
        if cls not in instances:
            instances[cls] = cls(*args, **kwargs)
        return instances[cls]
    return get_instance

@singleton
class MyBloatedConfig(dict):
    .... 

Is there some other more pythonic way to broadcast configuration changes across different modules?

Answer 1

The big, global (often singleton) config object is an anti-pattern.

Whether you have settings.py, a singleton in the style of MyBloatedConfig.get_instance(), or any of the other approaches you’ve outlined here, you’re basically using the same anti-pattern. The exact spelling doesn’t matter, these are all just ways to have a true global (as distinct from a Python module level global) shared by all of the code in your entire project.

This is an anti-pattern for a number of reasons:

• It makes your code difficult to unit test. Any code that changes its behavior based on this global is going to require some kind of hacking – often monkey-patching – in order to let you unit test its behavior under different configurations. Compare this to code which is instead written to accept arguments (as in, function arguments) and alters its behavior based on the values passed to it.
• It makes your code less re-usable. Since the configuration is global, you’ll have to jump through hoops if you ever want to use any of the code that relies on that configuration object under two different configurations. Your singleton can only represent one configuration. So instead you’ll have to swap global state back and forth to get the different behavior you want.
• It makes your code harder to understand. If you look at a piece of code that uses the global configuration and you want to know how it works, you’ll have to go look at the configuration. Much worse than this, though, is if you want to change your configuration you’ll have to look through your entire codebase to find any code that this might affect. This leads to the configuration growing over time, as you add new items to it and only infrequently remove or modify old ones, for fear of breaking something (or for lack of time to properly track down all users of the old item).

The above problems should hint to you what the solution is. If you have a function that needs to know the value of some constant, make it accept that value as an argument. If you have a function that needs a lot of values, then create a class that can wrap up those values in a convenient container and pass an instance of that class to the function.

The part of this solution that often bothers people is the part where they don’t want to spend the time typing out all of this argument passing. Whereas before you had functions that might have taken one or two (or even zero) arguments, now you’ll have functions that might need to take three or four arguments. And if you’re converting an application written in the style of settings.py, then you may find that some of your functions used half a dozen or more items from your global configuration, and these functions suddenly have a really long signature.

I won’t dispute that this is a potential issue, but should be looked upon mostly as an issue with the structure and organization of the existing code. The functions that end up with grossly long signatures depended on all of that data before. The fact was just obscured from you. And as with most programming patterns which hide aspects of your program from you, this is a bad thing. Once you are passing all of these values around explicitly, you’ll see where your abstractions need work. Maybe that 10 parameter function is doing too much, and would work better as three different functions. Or maybe you’ll notice that half of those parameters are actually related and always belong together as part of a container object. Perhaps you can even put some logic related to manipulation of those parameters onto that container object.