Question

I have an interesting log file format and I would like to parse it into Python for analysis.

The format is key=value with each separated by tabs, and a newline at the end of each entry, like this:

date="Mon, 04 Jul 2011 05:05:45 GMT"    addr=127.0.0.1  response_time=13    method=GET  url=/   status=200  referrer=   user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.112 Safari/534.30"

Now some of the fields may be changing, so I want the code to be flexible with what is thrown at it, as long as it is in the form of key=value key=value(etc.)

As of now, I have two embedded for loops, one to split each line into the key=value fields, and the other to split the key=value into their own separate entities.

Does this seem like the best way to go or is there a more elegant solution?

Answer 1

Two for loops seems fine for this problem. If I were coding it, I’d probably do something like this:

with open('log_file') as f:
    for line in f:
        fields = line.split('\t')
        for field in fields:
            key,_,val = field.partition('=')
            # Do something with each key and val