Question

I have a message definition file that looks like this

struct1 
{
  field="name" type="string" ignore="false"; 
  field="id" type="int" enums=" 0="val1" 1="val2" ";
}

struct2
{
  field = "object" type="struct1";
  ...
}

How can I parse this into a dictionary with keys ‘struct1, struct2’ and values should be a list of dictionaries, each corresponding to the respective line number so that i can do the following

dict['struct1'][0]['type'] // Would return 'string'
dict['struct1'][1]['type'] // Would return 'int'
dict['struct1'][1]['enums']['0'] // Would return 'val1'
dict['struct2'][0]['type'] // Would return 'struct1'

and so on..

Also, I can change the format of the definition file and if any of you have suggestions on modifying the definition file format to make it easier to parse, please let me know.

Thanks

Answer 1

I would simply use Python for the message definition file format.

Let your message definition file be a plain Python file:

# file messages.py
messages = dict(
    struct1=[
        dict(field="name", type="string", ignore=False),
        dict(field="id", type="int", enums={0: "val1", 1: "val2"}),
        ],
    struct2=[
        dict(field="object", type="struct1"),
        ]
    )

Your program can then import and use that data structure directly:

# in your program
from messages import messages
print messages['struct1'][0]["type"]
print messages['struct1'][1]['type']
print messages['struct1'][1]['enums'][0]
print messages['struct2'][0]['type']

Using this approach, you let Python do the parsing for you.

And you also gain a lot of possibilities. For instance, imagine you (for some strange reason) have a message structure with 1000 fields named “field_N”. Using a conventional file format you would have to add 1000 lines of field definitions (unless you build some looping into your config file parser – you are then on your way to creating a programming language anyway). Using Python for this purpose, you could do something like:

messages = dict(
    ...
    strange_msg=[dict(field="field_%d" % i) for i in range(1000)],
    ...
    )

BTW, on Python 2.6, using named tuples instead of dict is an option. Or use on of the numerous “Bunch” classes available (see the Python cookbook for a namedtuple for 2.5).

EDIT:

Below is code that reads message definition files as specified on the command line. It uses execfile instead of import.

# file mainprogram.py

def read_messages_from_file(filename):
    module_dict = {}
    execfile(filename, module_dict)
    return module_dict['messages']

if __name__ == "__main__":
    from pprint import pprint
    import sys

    for arg in sys.argv[1:]:
        messages = read_messages_from_file(arg)
        pprint(messages)

Executing:

$ python mainprogram.py messages1 messages2 messages3

will read and print the messages defined in each file.