Question

I’m trying to implement a templated configuration file.
I’d prefer python, but I’d take an answer in perl too.
I’ve used perl for my example.

I’ve searched a bit and found
– python single configuration file
– ConfigObj
– python configuration file generator
– ePerl
but I could not from those solve my problem.

I’m trying generate a configuration file mostly in the INI format (with not even sections):

# Comments
VAR1 = value1
EDITOR = vi

and I need that generated from a template where I’m embedding a scripting language inside the text:

# Config:
MYPWD = <:   `pwd`  :>

The text in between the ‘<:’ and ‘:>’ would be in the scripting language (python or perl). As with a template, its stdout is captured and inserted in the resulting text. The templating used in the example is basically eperl, but I’d prefer python if available.

and finally, the defined variables should be reusable:

# Config:
CODE_HOME = /some/path
CODE_BIN = <:=$CODE_HOME:>/bin

Here’s the test source file that I read in:

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# platform.cfg
# This one variable
VAR =value
# this is a templated variable. The langage is perl, but could be python.
HELLO= <: print 'World' :>
# This is a multi-line code which should resolve to a single line value.
LONGER = <:
 if (1) {
    print "abc ";
 }
 $pwd = `/bin/pwd`;
 chomp($pwd);
 print $pwd;
:>
# Another one to test the carriage returns.
MULTIPLE = /<: print "proj" :>/<: print "tahiti":>/<: 
print "pd/1/";
$system = `grep -w VAR platform.cfg | egrep -v 'print|platform.cfg' | cut -d = -f 2-`;
chomp($system);
print $system;
:>
# variables dependent from the previous variable definition
VAR1 = <: print $VAR :>1
# variables dependent from the previous variable definition
VAR2 = <: print $VAR1 :>2
# variables dependent from the previous variable definition
VAR3 = <: print $VAR2 :>3
# variables dependent from the previous variable definition
VAR4 = <: print $VAR3 :>4
# BTW, multi-line comments are significant
# and should be preserved as the documentation for the
# variable just below:
VAR5 = <: print $VAR4 :>5
VAR6 = <: print $VAR5 :>6
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

And I’m looking to get this result out of the script.
I could not figure how to have the variables defined in the config file be part of the interpreter?

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# platform.cfg
# This one variable
VAR =value
# this is a templated variable. The langage is perl, but could be python.
HELLO= World
# This is a multi-line code which should resolve to a single line value.
LONGER = abc /src/byop/CODE
# Another one to test the carriage returns.
MULTIPLE = /proj/tahiti/pd/1/value
# variables dependent from the previous variable definition
VAR1 = value1
# variables dependent from the previous variable definition
VAR2 = value12
# variables dependent from the previous variable definition
VAR3 = value123
# variables dependent from the previous variable definition
VAR4 = value1234
# BTW, multi-line comments are significant
# and should be preserved as the documentation for the
# variable just below:
VAR5 = value12345
VAR6 = value123456
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Thanks for your suggestions.

Answer 1

If you don’t mind using a different syntax there are several template libraries you could use, mako is similar in spirit, Jinaj2 is also pretty nice. Go with a tried and tested library! If you really want to implement your own template library this might give you a start:

import re, StringIO, sys

def exec_block(block, variables):
    """Captures output of exec'd code block"""
    code = compile(block.strip(), '<string>', 'exec')
    _stdout, result = sys.stdout, StringIO.StringIO()
    sys.stdout = sys.__stdout__ = result
    exec(code, variables)
    sys.stdout = sys.__stdout__ = _stdout
    return result.getvalue()

def format_template(template):
    """Replaces code blocks with {0} for string formating later"""
    def sub_blocks(matchobj):
        """re.sub function, adds match to blocks and replaces with {0}"""
        blocks.append(matchobj.group(0)[2:-2].strip())
        return '{0}'

    blocks = []
    template = re.sub(r'<:.+?:>', sub_blocks, template, flags=re.DOTALL).splitlines()
    blocks.reverse()
    return blocks, template

def render_template(blocks, template):
    """renders template, execs each code block and stores variables as we go"""
    composed, variables = [], {}
    for line in template:
        if '{0}' in line:
            replacement = exec_block(blocks.pop(), variables).strip()
            line = line.format(replacement)
        if not line.startswith('#') and '=' in line:
            k, v = [x.strip() for x in line.split('=')]
            variables[k] = v
        composed.append(line)
    return '\n'.join(composed)

if __name__ == '__main__':
    import sys
    with open(sys.argv[1]) as f:
        blocks, template = format_template(f.read())
        print rend_template(blocks, template)

Which basically works like the above, except uses Python for the code blocks. Only supports one block per assignment, which actually seems like the best approach to me. You could feed it a configuration file like:

VAR = value
LONGER = <:
    print 'something'
:>
VAR1 = <: print VAR :>1
# comment
VAR2 = <: print VAR1 :>2
VAR3 = <: print VAR2 :>3
VAR4 = <: print VAR3 :>4

And it would exec each block render the variables out for you:

VAR = value
LONGER = something
VAR1 = value1
# comment
VAR2 = value12
VAR3 = value123
VAR4 = value1234